There currently seems to be no shortage of startups in NYC, even without these tax breaks. And if a company only comes to NY because of the tax breaks, they might leave when the tax breaks expire.
I had the same thought until I realized that Human capital is a lot harder to move than buildings. Yes, it's possible to move a company away from a city, but 10 years is a long time. Long enough that a company would lose a ton of employees by moving out of state.
Make taxes higher, I say. I hope to one day have a New York City entirely funded by its full-time residents, so we can stop pretending to care about tourists.
A huge percentage of full-time residents work in the tourist industry (e.g., hotels, theaters, museums, etc.). And a lot of NYC business sectors would be much smaller without the tourist population, like restaurants and retail stores. Without the tourists, there would be a lot of unemployed New Yorkers.
Where I live (The Netherlands) that would be 52%, so $78,000. Sweden is even worse with 57% ($85,500), but the Swedish have way better incentives like (almost) free education/healthcare etc.