The real asshole is the one telling people how to live and making assumptions about other people's intentions because of the state of their property. If you are trying to sell an empty lot where a building used to be for example, you can't just start planting tomatoes. You also can't let random people enter your property because you'll be liable for their injuries.
There is usually no technical reason why you can't, but any vegetation needs labor to maintain, a source of water, soil that is actually suitable and not poisoned. Then this vegetation would make the property hard to visually see or traverse, and it would likely be destroyed by the buyer anyway. There may also be zoning restrictions against growing tall vegetation on random land, leading to fines and fees. If you create visual obstructions, homeless people might also start squatting on the land or littering, leading to further headaches.
As cool as it sounds to just whimsically plant flowers, vegetables, and trees everywhere, this does create problems for people. Just because someone has property and you don't doesn't mean you have a divine right to tell them what to do with it. It could literally be owned by an old granny who is trying to sell it with the least amount of headaches, or someone who intends to build on it as soon as they save enough money to do so.