Simple case that Waymo etc still struggle with is making a left turn through an intersection or an unprotected left. Even something as common and predictable as crossing oncoming traffic that humans do every day is still causes disengaments for these cars. Cruise has a team just focused on issues at intersections. Throw in some pedestrians and this is very hard. Humans can intuit a lot from a pedestrians behavior that these cars just cannot handle. Is someone standing by the corner or about to walk into the street? People do a lot of weird unpredictable things in downtown areas. Tesla has done ok with the easiest part of driving, freeways, with the support of an attentive human but three are still many reported problems. Moving on to complex environments with multiple moving actors at such high reliability that it is a hands free production-grade system in 1-2 years is not likley, but good luck to them.
Isn't that the exact type problem that massively benefits from having tons of real world data because they will have examples to train their neural nets for practically every possibility?
Not really. Neural nets are good at detection and classification problems for objects in the environment but they are not a panacea. Assuming a neural net is good enough to detect everything a car will see (for pedestrians this isn't the case) path planning while predicting behavior of other actors is still a very difficult unsolved problem.