That's a huge it depends question. I think it is a delicate balance, and what that is depends on what is being weighed. I am assuming that this is your opinion, too, and want to flesh the question out using the massive processing (*cough* bullshitting *cough, cough*) power here on permies. This is my take on it.
So you have your property taxes on one side, along with all the costs associated with actually making use of it, which often requires some material outlay, such as for a dwelling or renovation thereof, equipment specific to any operation being undertaken on the property, and the associated outbuildings, not to mention infrastructure for access and animal barriers. I like to keep in mind that this is not an exhaustive list.
On the other side is the cashflow you can generate off that land. What you can do on that land, practically speaking, is limited by your resources on that land, unless you're willing and able to bring stuff in, say to amend soil, which becomes a more expensive and difficult option where it comes to broad-acre operations.
You'd know better than I, Travis, but I would say that along with soil and water conditions, climate is a large deciding factor in terms of what can be done where, and how great the yield. I think that more options are available if, as you have said in other posts, you properly evaluate your land to choose the most site-appropriate operations so you aren't fighting your land.
One other thing that you have brought up in other posts is using the socio-political landscape to your advantage, and the legislative one. Applying for grants for specific best-practices or tax-breaks for this and that protective environmental feature can tilt the balance in your favour, if it doesn't negatively impact your operations on the land, such as if grants for wetlands protection and riparian stewardship translated to not being able to use it for wildcrafting or carefully managed selective rotational grazing.
I am in an interesting position, where I am living in something like the second-most expensive city in which to live in Canada and working a job, like a lot of us, that pays not enough money, trying to pay down my debt, and looking for options out of the city where I could work a full-time job, ideally carry a mortgage on a house with a bit of land, and carry on with carrying on.
For me, my ideal homestead size for right now would be the largest, best property I could afford now, or soon. I might have to compromise, too, and rent a dwelling out of the city, and find a bit of land to rent, or rent-to-own, ideally, on which to run a market garden and animal fibre or meat operation, unless or until I can buy some outright.
In terms of practical numbers, I think J.M. Fortier, who, if you aren't familiar, applies permacultural techniques to intensively managed market gardens and (as far as I can tell) uses only unpowered walking-plow-scale garden machinery, said that the largest market garden he could operate by himself, in the style he operates, was something like 3/4 of an acre. I don't recall how much food he was producing, or if the focus could be shifted to provide a mix of in-season garden produce and produce that overwinters well.
For scale, I believe that it was generally acknowledged that potatoes grown in poor soil could feed two adults per acre per year.
And then there are orchard crops to think of.
Without going into absolutely everything, I think that it will depend on how much of your property-derived income is outer-zone operation, usually needing more space to be profitable, and how much will be inner-zone, intensive operation. I think the more intensive operation you have, the smaller the property can be, to a point, but that has to take into account what type of work and how large a workload you can or want to handle.
Honestly, I think that you have to go with your experience. That is the path of least stress. If you are comfortable with the sort of work you do with large equipment and property conversion, maybe you need to be looking at 15 acres, or maybe larger. If it's more taxing for you to operate a half-acre intensively-managed market or kitchen garden than it is to convert forest to pasture for sheep, say, the ideal is the larger quantity.
Sorry for the ramble. I have been going over this for myself a lot, and it sometimes helps to put it down in text.
I hope some of this is in some way useful.