According to https://www.curbed.com/2018/12/17/18144657/construction-home..., San Francisco's high construction cost is mainly driven by the high cost of construction labor. It is disproportionately driven by labor cost when compared to all other cities.
So we have some data now, rather than just pithy statements, and I don't think it agrees with what you want.