As an aside, a big reason existing residential homes are often profitable speculative investments is because of bans on new homes. The lack of competition from new home construction drives up the price of older homes.
It isn't just new homes, it is also replacing old homes. If I cannot tear down a house next to downtown because it is "historical" (because it is old, not because anything historically important happened there), then I can't replace it with an apartment and in turn that forces more people to move out to suburbs.