My understanding was that the actual ground in NYC roughly determined building height. Manhattan was originally a swampy island, so you need to find something hard underneath (bedrock) to build on if you want 50 stories.
Depth to Bedrock and the Formation of the Manhattan Skyline, 1890–1915
New York City historiography holds that Manhattan developed two business centers—downtown and midtown—because the bedrock is close to the surface at these locations, with a bedrock “valley” in between. This article is the first effort to measure the effect of depth to bedrock on construction costs and the location of skyscrapers. We find that while depth to bedrock had a modest effect on costs (up to 7 percent), it had relatively little influence on the location of skyscrapers.
This is actually something of a myth [1].
[1] https://www.google.com/amp/observer.com/2012/01/uncanny-vall...