To expand on this a little more, the absence of accountability contributes to the loss of learning. Mistakes and errors will always happen, whether they are sourced by humans or machines. But something (the human or the machine) has to be able to take accountability to have the opportunity to learn and improve so the chances of the same mistake happening again go down.
Since machines don't yet have the ability to take accountability, it falls on the human to do that. And organizations must enable / enforce this so they too can learn and improve.
Without that, there's a lot of dependency being pushed on the machine to (cross fingers) not make the same mistake again.
> The problem is that people are now building our world around tooling that eschews accountability.
Management has doing a wonderful job of eschewing accountability for decades.
It's a lot of people's dream to be able to say, yeah, our product doesn't work, but it's not OUR fault, and the client just shrug and grumble ai ai ai, and just put up with it because they know they can't get a better service anywhere else.
It's not MY fault my website is down: it's Amazon's! It's not MY fault my app doesn't work: it's Claude Code's!
Well just to be clear from a legal perspective, in the case of AI, as long as AI is "property", the owners, developers, and/or users will be held liable for things like the hypothetical fatal car accident that Sussman posits.
Currently, from a legal perspective, AI is considered a "tool" without legal persona. So you sue the developer, the owner, or the user of the AI. (Just kidding, any lawyer worth his/her salt will sue all three! But you get the point.)
Legally speaking, AI will probably be viewed that way for a long time. There are too many issues agitating against viewing it any other way. Owners will not give up property rights. No will to overbear. On and on and on.
>complex systems are a pretty good shield from accountability in practice today.
Maybe complex legal systems are, but complex software systems offer you no such protection.
My field for the past few decades has been diagnostic medical software. In that field, the 501K you got is kind of entering you into an ironclad agreement with the government. There's almost no way out of it. 501K certs significantly simplify, (for the government), holding you accountable. You have made attestations to suitability directly to the federal government. And the way our chief counsel explained it to us, literally each signature you sent to the government, for each feature that failed, is actually a single count of lying to the federal government.
Please, please, please people, don't listen to comments like the one above. Everything should be run by your qualified legal expert. Getting things right up front is so much easier than trying to fix things when the inevitable happens.
Alternatively, stick to fields free from regulation. That's also a viable strategy. But to just trust that the legal system is complicated and the technology you're deploying is complicated, so the feds will never get me? That's the start of a lot of really bad stories.
I don’t think it’s missing, I just think it’s seen as a liability, and American society has been known to absolutely obliterate people who are liable.
Everyone thinks they have the right to judge, and use the massive amounts of available information to do so, even if they haven’t been trained to judge.
We dont know the final amount, as they settled out of court, but in 1992 a woman was awarded hundreds of thousands of dollars by the judge after receiving third degree burns from a coffee at a McDonalds.
She had originally asked for $20,000 to cover medical expenses.
If instead this happened in another part of the world instead of the USA, I doubt that McDonalds would have had to pay much if anything in a similar situation.
And the point is that it seems that especially in the USA the companies are very avoidant of ever admitting fault for anything happening to their customers, for fear of lawsuits where they have to pay a lot of money to individual people.
This is such a litmus test, this case. Yes, America does weird things with punitive damages. But the injuries were really severe and the negligence significant. More often you get class action lawsuits where everyone involved gets mailed a cheque for $3.
It's not just America. McDonald's UK got involved in the UK's biggest ever libel case. https://en.wikipedia.org/wiki/McLibel_case ; leaflets distributed in 1985 ended up resulting in a human rights judgement in 2005, after a lifetime of litigation and millions spent.
Seems kind of an opposite situation. There it was McDonalds suing a pair of people, not the other way around. And the human rights violation was by the UK government and not McD.
I think it's about sending a message, and maybe also about ego to a lesser extent. If you pay the 20,000 you're admitting you did something wrong. Of course they did do something wrong. They did something wrong thousands, probably millions of times. And they got very lucky to only be sued once. Which is why they paid millions.
Yes, McDonals paid one injured party out of many who may - whose injuries were not big enough or who did not have the time/money/energy for a lawsuit but whose combined damages could easily be more than hundreds of thousands.
In socialist countries we have understood that it is better for the individual that everyone pays a share of what it costs to maintain a functioning healthcare system, and making it available to everyone for free.
Since machines don't yet have the ability to take accountability, it falls on the human to do that. And organizations must enable / enforce this so they too can learn and improve.
Without that, there's a lot of dependency being pushed on the machine to (cross fingers) not make the same mistake again.