Would you oppose a smelly wastewater treatment plant being built on the previously residential property right next to you? That's why they should get a say.
Is it not? I think it's very comparable, you attempted to appeal to the freedoms of property owners but there are laws the govern the use of land wherever you go.