Yes? It drives up housing costs for everyone else. Which may sort itself out in the very long term (on the scale of decades), with sane people moving elsewhere and siting companies elsewhere, but in the shorter term you have a finite number of job opportunities, and (except for remote workers, obviously) you have to live one of those places, and every unoccupied residential property forces you to live farther from work or pay more (reduced supply).
Seems like it's mostly stuff like bans on multifamily buildings, mandatory wasted space between the home and the sidewalk, mandatory parking spaces, and multi-year permitting processes that drive up housing costs.
There's no inherent reason why "job opportunities" have to be in only a very few places.
I'm not sure anyway that local workers are fundamentally more entitled to be allowed to buy property than other groups of people who might want to spend money to acquire real property.
Okay, but these opportunities still are in only a very few places and housing is a necessity. A human right, if you will. So the issue of housing needs to be addressed in all ways possible, including making sure that properties aren't left off the market.
You should note that "owning" a property that you don't live in (or rent out) also requires the use of force -- state force to remove anybody attempting to actually use the space.
I suspect you're trying to make a pro-liberty / property rights point, in which case I think you should consider more carefully the basis of justified property ownership.
Even if you are inside it 100% of the time, it requires violence or the threat of violence to defend against trespassers that might just try to violently throw you out.
Violence is an inherent aspect of human affairs, so the question in every case isn’t whether violence is necessary but rather whether it’s justified.