[bko]

> This comes across as them selling a product they know could fail in dangerous ways, but they don't want to be responsible for any of it.

This is just a safety precaution. Why wouldn't they put this in there? It may not hold up in court but it can't hurt. I don't think this means they "know it could fail in dangerous ways".

The safety model in comma.ai is actually quite brilliant. It can't perform any action faster than you're able to correct and disengage. To test it, they have someone drive while a malicious passenger seat has full access of the controls as limited to by the software. The passenger then messes with the steering and acceleration without the main driver's knowledge. The driver has to prevent the actions. The torque limit is much lower than that of Tesla or other lane-keep assist tools.

[picks_at_nits]

"It may not hold up in court but it can't hurt"

If you sell someone something with a nudge-nudge, wink-wink, and they get killed using it, it absolutely hurts. You may be able to weasel out of being held accountable for it, in which case it won't hurt you, but the larger issue here is that this kind of misleading copy can lead to people making poor decisions.

You may have put it in the fine print that it's not a real product, but the whole point of nudge-nudge wink-wink is to strongly imply that it's a real product worth real money, and thus you are going out of your way to encourage people to try it and take chances with real lives.

[bko]

What's the appropriate level of liability?

If I buy a cell phone holder for my car, and it distracts me and I get into an accident? What if Car Play lags and i'm distracted and I get into an accident? What if radio plays an ambulance and I freak out and get into an accident? What if my sunglasses make me mistake a red light?

This product does lane assist. It does a good job according to consumer reports [0], higher than all other lane assists. It doesn't detect stop signs or traffic lights or drive for you. It keeps your lane. It acts predictably and gives the driver enough time to react.

Unfortunately the liability model is messed up. I think this product is relatively tame and should allow to exist. And you need to pay attention. They even have inward facing cameras to make sure you're paying attention, more than most other companies. They do everything they can to be safe but of course they're not stupid and they'll put in a sweeping statement on liability.

This is really pushing forward the self-driving industry and is an incredible feat of engineering. It's much more open and transparent than every other lane keeping software, and it's being developed with a lot of thought and care from a talented engineer as opposed to some nameless faceless bureaucratic commission in Ford or some other dinosaur.

https://www.thedrive.com/news/37833/consumer-reports-ranks-t...

[picks_at_nits]

I'm not gonna debate the "appropriate level of liability."

My point has to do with what you're signalling. If a thing is alpha-level, and real humans can get killed, I wouldn't let random people buy it and use it in their cars, period.

Informed consent is deeply problematic for a product like this: Very few people have the expertise to look at the code and the hardware and properly evaluate the risks, right down to understanding which kinds of edge cases need to be very carefully avoided.

Unless you're vetting researchers and barring people who just want to save a few bucks and brag their car self-drives, you really don't know if every person who downloads the extra software really does grasp the implications of what they're consenting to.

You might grasp the implications, and so might many people in this thread, but that doesn't guarantee that everyone does. THE AUDIENCE OF HACKER NEWS IS NOT A REPRESENTATIVE SAMPLE OF SOCIETY.

And we are talking about a product to be used on open roads: In addition to informed consent from the person who downloads the software, if they get into an accident with another vehicle, pedestrian, or cyclist, did any of those people consent to share the road with someone who installed alpha software on their device?

Morally, I can't get behind a few disclaimers and a nudge-nudge, wink-wink for any kind of autonomous driving tech, even if it's "just" lane-keeping.

———

Update: But to be clear, I am in favour of people tinkering with all sorts of digital automotive tech, and we really should find a way for lone inventors or small teams to innovate without the "enterprise outfits" using regulatory capture to drown small competitors with red tape.

I'm only arguing in favour of truly informed consent, which I believe is tricky for driver assistance technology being provided to arbitrary customers.

[bko]

So your main problem is about the disclaimer and that its called alpha. I provided a source that rates it the best product among all other competitors and the highest score on keep driver engage. And they have the most miles of any other lane assist technology. So I think its safe. I think the alpha is more tongue in cheek and is not a term that means anything really apart from, as you say, a wink and a nod.

For the laymen user, they won't read the disclaimer or understand what Alpha means or even know that is is "alpha". I'm an engineer and I probably won't ever really audit the code. I will do my research like most other people, read online reviews or testimonials like Consumer Reports.

So are you against all lane assist technology? How about auto-braking? Anti lock breaks?

[picks_at_nits]

I'm against just heaving that technology out over the fence into the hands of consumers and leaving it up to consumer reports and/or individual consumers to decide if it's safe enough.

Safety is a 'picking up nickels off of railway tracks" problem. A thing might work 10,000 times in a row, but then suddenly, catastrophically fail because something is different that hadn't been tested before, like dealing with a woman walking her bike across a multi-lane road.

this is not a good scenario to leave up to consumers to decide whether a thing is safe. Not even with consumer reports to help out with testing.

Now as to ABS, the comparison is not even close. I do not buy ABS by purchasing brakes and then flashing some ROM with code I download from the internet. ABS is covered by all sorts of regulatory frameworks around the world, it isn't simply cooked up and offerred for download like it's an MP3 player skin.

Even though it's a much more mature technology, the problem with ABS is again, consumers cannot give informed consent to a disclaimer when purchasing it from some random person.

When I buy it as part of an automobile from a manufacturer that complies (I'm looking at you, VW) with regulations, I'm consenting to trusting something in a completely different way than when I download code and there's an MIT license or whatever weasel-wording somebody em ploys to say, "If you die, sucks to be you. If you kill someone, it's your soul that will be in torment."

Your equivocation of 1. downloading code for a safety feature from the internet that's marked "alpha" and has been tested according to whatever the author feels like testing because it's not offered as a "product," with 2. purchasing an automobile that has ABS brakes which are tested and maintained within a global safety regulatory framework...

You're entitled to whatever workdview you like, but on this pointI believe our discussion ends. There is a fundamental axiomatic belief I hold that is not compatibvle with a fiundamental axiomatic belief you hold.

I don't want to spend all day trying to explain why I believe Volvo selling a three-point harness is not the same as some random person knitting a seta -belt, selling it on etsy, and leaving it up to you and I to read the consumer reviews to decide whether it's safe enough.

You believe the free market plus informed consumers will sort all this out. I do not.

[jeffreygoesto]

Please don't doxx Ford engineers if you don't give any proof. There are hard working, ethical people working who don't want to kill people by lightheartedly pushing stuff on the road. Just because you don't know them does not mean they are not talented.

[greenrd]

I think you meant "diss", not "doxx".