I'm a programmer and I daydream about doing landscaping or being a mechanic or renovating houses. I spend the weekends working in the yard or on my cars/motorcycles or my house. There's just not enough pay in something like that, at least in the city. Maybe some day I'll move back to my small hometown and start a business in one of those areas. If I didn't have to make as much money, I'd do a job like that. Physical labor is good for the soul.
You've just got to dream bigger. Figure out how to do it better than the traditional model of either 1) doing it all yourself, or 2) hiring cheap labor to do it for you. And got God's sake, hire some analytics help so you understand how much help you need, what you can afford to pay, how much demand you've got (and when), etc. But yeah, I hear you: the six years I worked at the airport back in college were some of the best working days of my life (I worked at a private FBO). You're outside all the time, moving, physical active... You see sunrises and sunsets, work in the rain, snow, and scorching heat. And you mentally and physically feel much better than being chained to a desk.
I think you'd be surprised about the pay. Landscaping and renovating can pay very well. The trick is that you need to keep the work coming in. I have lots of friends in the construction industry and they are basically all nerds at heart. They are motivated by the technology and building/fixing things, not dissimilar to a programmer. But with any job you have to move up to management. If you can't exploit technology to do the job then the physical demands can bear down on you.