The lack of housing is due to incentives. Investors aren't going to increase supply and in return decrease their margins, they'll just make their money work somewhere else. Free markets work against us here, this sort of thing has to be addressed via price-fixing/subsidies.
The incentives are aligned for builders (who want to build housing for money) and buyers (who want to pay money for housing). What's getting in the way is overly restricted zoning, including single-family-unit only, and minimum parking requirements.
The free market is the solution here, what's holding it back is zoning.