| > If you think it's a black box to the Tesla drivers, how is it not a black box to the Waymo customers in the back seat of these cars? The general public (as a vehicle occupant) only interacts with a Waymo vehicle as a passenger with no vehicle control responsibilities. That is in stark contrast from the integral human-machine relationship that exists in a Tesla vehicle. > If you mean to the teams, you cannot assume that Waymo's systems are any less of a black box than Tesla's systems. True. Waymo's internal processes are a Black Box to me (and anyone external to Waymo) because we are not read into their systems safety lifecycle, whatever it may be. Hopefully and presumably, Waymo is maintaining a Safety Management System (SMS) with their test operators and other internal teams, as they have claimed in the past. Of course, since there is little-to-no regulatory oversight of this in the US (at the moment, perhaps)... Waymo's "word" is really the only thing the public has to go by. That is not acceptable, in my view, in constructing a novel transportation system that ultimately relies on public trust to be economically viable... but that is the regulatory reality right now. In the case of Tesla, it is definitive that they are not maintaining a SMS, in large part, because Tesla's (untrained) customers utilizing the system cannot be sufficiently read into a lifecycle. There is simply no way to do that without maintaining a highly-controlled, continuous relationship with the test operator. For example, the "release notes" (sprinkled with some Tweets from Musk) that Tesla issues with some of the FSD Beta updates are simply too puny relative to the complexity of not only the vehicle system, but the larger complexity of the roadway. > And even then, they're not much of a black box at all, besides the actual object detection, as both Waymo and Tesla still have most decision-making in regular logic-based code, not machine learning algorithms; and when they do, such as with "do I need to get over now to make the next turn", it's still fed back into the "business logic" that decides what to do and thus logged and audited when it's sent back to HQ. As I stated elsewhere, these are physical safety-critical systems where the totality of the systems safety components cannot be expressed in software alone. Remote vehicle telemetry is valuable of course, but as a tool to serve the validation process... not the validation process itself. Vehicle telemetry cannot be a complete accounting of all of the interacting systems safety components involved here. For that, like all other safety-critical systems, one needs exhaustive, controlled and physical validation. |