Rising compared to what? Junior engineers are doubling or tripling up to live near work in metropolitan areas or face crushing commutes. It's a sign that even in software engineering, capital is crushing labor.
You're describing two different problems. Housing supply in the Bay Area and New York -- the areas I'm assuming you're talking about, because most other metropolitan cities are building to keep up with supply just fine -- is incredibly low. In addition, in the Bay Area especially, the incredibly high salaries in combination with that supply is causing skyrocketing prices.
Not really, there's cities all over the world with conditions that lead to crazy high real estate/housing costs. Most every metropolis ends up with a similar situation. Slightly different settings that all sum up to "high demand low supplies". NYC, London, Vancouver, Hong Kong, Tokyo, ...
Is this true when you are comparing similar conditions? Something along the lines of:
Large cities with significant multiculturalism, high tech jobs, significant culture (arts, media, entertainment, museums, etc.), public transportation, and openness to various ways of life
Yes. Try moving to Minneapolis, which has plenty of everything you ask for and housing prices are not out of control. If you don't need a large city, Des Moines does well too on the rest of your qualifications.
Not Compared to London though - London does have a much better transport system so living 60-70 miles away and commuting isn't as bad as it would be in SV.