[pasabagi]

My gut feeling is that even if you did remove distortions, the problem would persist. If the price for a commodity ultimately stabilizes at material cost + labour, I think rent ultimately stabilizes at 'everything you can afford to pay and some'.

Ultimately, commodity price is driven down by the fact there's always some industrialist who can flood the market with cheaper crap, until there's basically no profit in it. You can't flood the market with 'living in walking distance from work', and a large part of the cost of any building project is simply buying the plot to build on, because each landowner has no direct competition.