[jefft255]

I think you're mostly right; the only thing I'm more skeptical about is the availability and reliability of high def maps and semantic knowledge as you say. Maintaining such thing will be expensive and I wonder how it will scale country-wide as opposed to a few select cities. I bet using the self driving cars themselves to do the mapping and correct mistakes will be a big part of that.

Rough localization in a snowstorm (as in good enough to use prior knowledge of signs) or rain shouldn't be too hard. GPS still works in bad weather and lidar localization (scan-to-map matching) should be hampered but still usuable.