Unrealized losses. The risk, of course, is not zero, but you only really lose if you're forced to sell. In the meantime, you need a place to live, either way. Not a good gamble if your time horizon is less than 7-10 years, to be sure.
I'm not sure why "unrealized losses" would make anyone feel better? People need to move, get divorced, lose their jobs. "Just stay in the house and keep paying the mortgage" is not an alternative available to everyone.
Look at Las Vegas. It only recovered to the 2008 peak last year. It's been a terrible investment.
This is more of an argument against investing at all than it is specifically about housing. Obviously, if you want upside, then you have to tolerate some level of risk. If the argument is just, "things could go badly in your life at exactly the time you're in the red," then that's true of anything.