Yes, this is famously true, with SF's cost of living (especially but not only housing) being some of the very highest in the nation. Getting out of California helps even more.
Well they can. If I move to another popular place like Manhattan they probably won't go down much. But it's reasonable to ask, if you don't have a work-related reason to live in a place, whether it's good value based on your priorities. The answer may be "Yes!" for SF for some people. But it may also be "No."