[skybrian]

You're misunderstanding the argument. Hours worked can remain the same while production and wealth increase. There are many jobs this isn't true of, but there are enough where it is true that that it can drive up housing costs. (A few rich people wouldn't be enough to do that, so there have to be lots of people.)

A better counter to this argument is that national averages are misleading and irrelevant for many people.