Who will build the infrastructure (roads, electricity, water, sewage)? If you build sth without planning you can block or make it harder to provide these services to others. It only makes sense to plan such thing beforehand.
Planning restrictions are very popular with the local public. So much so that some places in the US invent their own weird private contract law versions (HOAs).