[kfarr]

Maybe it's just me, but the regulatory oversight around these automated features seems insufficient. I'm trying to imagine equivalents from older passive safety technology like seatbelts that don't always work, airbags that pop out prematurely, or cruise control that accelerates without warning. Doesn't seem like those would pass muster "back in the day"? (Not to mention Uber autonomous vehicle flat out killing a pedestrian with no criminal culpability.)

Maybe it's a case of severity -- automated lane guidance isn't viewed as a serious problem when it fails? But who makes that decision?

I don't drive much anymore but when I do it's in a Zipcar with relatively recent Subarus that have this lane guidance technology. I always turn it off as it invariably makes mistakes within a few hours of usage and seems more dangerous on than deactivated.

[looping__lui]

Yes. I was dead shocked when I took a brand new Audi rental car on the German highway. Per 150 miles driven I probably had one severe incident, including: sudden deceleration from 100mph to almost full stop with no car in front but a couple behind, the car actively steering out of the lane (construction site, straight lane), sudden deceleration when the car in front leaves the highway and slows down (but I stay on the highway), not detecting the giant SUV in front of me and almost rear-ending it. On a 400miles drive I usually have ALL of the above. I never had an accident in 20 years of driving. From developing software I know: your users may interpret the UX differently from you or your designers and communicating well with your users is hard. In these circumstances I am always thinking: like what the heck are those Volkswagen people thinking about how the 5% of people having a different interpretation of this user interaction will end up!? I mean, how often do people end up doing things with software nobody expects them doing? “Oh, lets give them a functionality that completely ‘out of the blue’ potentially kills them every 200miles”.

[numpad0]

Automotive peoples’ mindset of safety and computing is weird.

They say it’s better to floor it when leading car is not moving, or better to engage lane centering at a borderline safe moment, to make drivers used to it. Or they say they want computers to delay software edge-case issues after it had occurred.

They have passions but the whole industry don’t seem to have much background handling sentient beings.

[UweSchmidt]

Regulating technology is difficult. You can easily set up a mechanical seatbelt test that tests a mechanism 1000 times, but how would you test an algorithm? Also, many people died before seatbelts became mandatory; so much for "back in the day".

[carlmr]

>how would you test an algorithm?

You can send out drivers that test if it works well. And then you can record that data, and check what is causing the issues and add these frames as test data?

But the testing IMHO is a bit too late. You need UX designers and psychologists giving inputs from the start. You need SW quality by design, not by process. You need to rid yourself of any management layers that don't understand software.