It's not anywhere near a crazy plan. Many thousands of 'expats' live in central and south America. I, myself, plan to be one someday. Keep in mind that there are only two countries (outside of the carribean islands) that speak english: Belieze and Guyana. Brazil speaks portuguese and the rest speak spanish, altho French Guyana speaks french and Surinam speaks dutch.
If you watch HGTV, you will notice a lot of people buying houses at prices that rival nicer homes in the US and Canada. These people tend to be rather spoiled and want that millionaire lifestyle, which can be had, at a price. If you set your sight a little more frugally, you can find lots of reasonable properties that you can fix up over time.
I agree, growing fruits and veggies year-round is a priority for me, too, and to do it in an ecologically friendly manner is even more important.
Many adventures await you, some wonderful and some scary, such as insects you have never encountered before. Make local connections, and find out how they handle some of the issues.
Be prepared for two seasons, wet and dry. You might want to explore a bit in each of these seasons, so you know what to expect.
Here is an online magazine that you might enjoy: http://www.escapefromamerica.com/