And along with permitting/zoning BS, there is also many other headwinds for real estate: high aggregate income, land scarcity, automation challenges (no prefab houses), and high labor costs.
In the places people want to live, yeah. It would be great if Manhattan were twice as wide, for example. There is the artificial scarcity from the zoning stuff, but then there is also just the geographic boundaries and human preferences for the "cool part of town".
Manhattan already builds higher. It just so happens that that new real estate is being sold as multi-million dollar condos instead of affordable housing (for certain definitions of "affordable")