[red75prime]

> So what if your image classifier thinks a cat is an impala?

For starters, the artificial neural network is the only instrument we have for sufficiently robust image classification. If it doesn't work nothing else will. We can only validate and retrain.

Getting back to self-driving. What if an occupancy network misclassify an impassable surface patch as passable? It would mean a possible accident, so you bias the network towards impassable classification. In this way misclassification is overwhelmingly more likely to cause spurious braking, unneeded swerve, and the like rather that a collision ("brake stabbing" was a recurring problem in Tesla FSD. Today it is largely mitigated.)

> The fact they can't is the strongest evidence that whatever proprietary secret sauce moat they have... doesn't really work.

Well, Waymo has publicly embraced world models, VLMs and end-to-end backpropagation relatively recently (December 9, 2025)[1]. There's not enough data yet to judge how this has influenced their expansion rate. Waymo representatives tout improved generalization (sorry, I can't find the interview. I think it was Dmitri Dolgov.)

> if their systems were really autonomous they could just deploy them anywhere a human driver can drive

As I've said it's a matter of degree. Human-caused road casualties are a familiar and largely inevitable evil. An autonomous vehicle killing a person will (and already has) cause a media storm. They can't ignore a potentially problematic distribution shift caused by deployment in a new area.

You are talking about the perfect system. Real systems are computationally constrained. Distillation necessarily degrades performance and might require retraining for particular areas.

[1] https://waymo.com/blog/2025/12/demonstrably-safe-ai-for-auto...

[YeGoblynQueenne]

I largely agree with what you say, except the bit where you say I'm talking about the perfect system. I tried hard not to give that impression because that's not what I'm on about. Autonomous systems trained with deep learning (including Deep RL from human demonstrations and so on) are very far from "perfect". They can perform very well in a broad range of scenarios until they encounter an unforeseen situation and their performance falls off a cliff. The problem is that the real world is full of unforeseen situations, full of long-tailed distributions that are near impossible to learn just by training on data, or anyway a reasonable amount of data (even very big data).

Sure, you can "mitigate" whatever edge case you become aware of during operation but when the edge cases are the entire distribution there's always more of them where the ones you mitigated for came from and that's a big problem, it's a very real problem, and we don't yet know how to solve it.

>> Well, Waymo has publicly embraced world models, VLMs and end-to-end backpropagation relatively recently (December 9, 2025)[1]. There's not enough data yet to judge how this has influenced their expansion rate. Waymo representatives tout improved generalization (sorry, I can't find the interview. I think it was Dmitri Dolgov.)

Well, let's wait and see. But my intuition is that Waymo is simply trying the latest trend to see if it will change something. It might even move the needle a bit, who knows.