[anonym29]

>AI can't be held accountable

I hear this all the time. Why does it matter? Punishing a human for making a mistake does not prevent mistakes, nor does it undo the harm of the mistake. A human saying "my bad, I messed up" and an AI saying "my bad, I messed up" are equally worthless, in a functional sense.

[forgetfreeman]

"Punishing a human for making a mistake does not prevent mistakes" This statement suggests you don't believe in some combination of neuroplasticity as a concept or the arrow of time.

[anonym29]

Tell the families of the people who died on 737 MAX disasters. "Don't worry - everything's going to be okay! The engineers learned from their mistakes - accountability works, you have nothing to be sad about!"

Tell the family of the person killed by a semi truck driver who showed up to work drunk or high: "Don't worry - the driver went to jail! Accountability prevented anything bad from happening!"

Accountability alone fails to prevent deadly mistakes millions of times a day; millions of mistakes are avoided daily through process, redundancy, independent review, and formal methods.

"Accountability prevents mistakes" is a comforting delusion. In reality, accountability is only marginally related to whether or not mistakes are made.

[forgetfreeman]

"Accountability alone fails to prevent deadly mistakes millions of times a day"

...in his desperation to finally win an argument online our hero advanced, grimly ignoring the concept of Engineering.

"millions of mistakes are avoided daily through process, redundancy, independent review, and formal methods."

Ahh spoke too soon, Engineering has finally joined the chat. So what mechanism do you propose lead to the foundation of process, redundancy, independent review, and formal methods?

[russelldjimmy]

What are you even on about mate? Sure accountability doesn’t prevent all mistakes. Guess what, nothing prevents all mistakes. Accountability can help prevent some mistakes some of the time. It sounds like you’re suggesting getting rid of the concept of accountability because it doesn’t prevent ALL mistakes. Way to throw the baby out with the bath water.

[anonym29]

>It sounds like you’re suggesting getting rid of the concept of accountability

Where on earth are you getting this?

Accountability alone is insufficient, and the things that actually prevent mistakes don't require it. The mechanisms that do work: independent reviews, redundancy, formal verification where applicable, staged rollouts, testing against adversarial inputs... these all function on the artifact, not on whether or not the entity producing the artifact can be held accountable. A formally verified proof is correct whether a human or an LLM generated it. A code review catches the same bug regardless of who's liable for shipping it.

The argument isn't "let's get rid of accountability", the argument is "it's ridiculous to suggest that the reason you shouldn't use AI is that AI lacks accountability - lacking accountability isn't the reason AI makes mistakes, and adding accountability to AI won't prevent AI from making mistakes - the answer to preventing mistakes with AI rests in process, and accountability does nothing to inherently ensure that".

Accountability is nothing but a transmission mechanism, and is blind to the values instilled through it. Accountability is literally what caused the 737 MAX disasters. The FAA decided it was more accountable to industry efficiency than it was to safety when it allowed Boeing to self-certify, which violated a process control of independent reviews. Boeing's board decided it wanted to more accountable to shareholder value maximization than it was to safety when it allowed MCAS to experience scope creep without re-review, which violated a process control of formal verification. Boeing's designers and engineers decided to be accountable to shareholder value maximization when they decided to make MCAS rely on only one of two flight control computers, which violated a process control of redundancy. Engineers at Boeing flagged these failures, but they ignored when management decided to be more accountable to shipping on time, which violated a process control of incorporating adversarial inputs and feedback.

Accountability did not prevent these mistakes, it caused them. Failure to abide by process controls caused the mistakes. Adding more accountability wouldn't have prevented these mistakes; maintaining strict adherence to the process controls that used to be in place would have.

[legostormtroopr]

If a human messes up enough eventually they well get fired, fined or jailed. An AI will not.

A human also knows they might get punished if it messes up bad enough, which might cause it to think twice before doing something bad. For an AI there is a reward, but there is no risk.

So while both might lie, only the human will be worried that it will be found out. That makes a difference.

[anonym29]

You say that like all humans are alike: that they all care about getting fired, fined, or jailed; that they're even considering punishment when they're making their decisions; that risk factors into decision making.

What you are describing is a hypothetical "rational person". In real life, even the most rational people you know do completely irrational things routinely.

The Therac-25 engineers were accountable. The 737 MAX engineers were accountable. Accountability is doing much less work in the safety story than you seem to think.

The real work is done by process, redundancy, independent review, formal methods. None of these inherently require someone to be penalized for making mistakes, and penalizing people for making mistakes is a demonstrably, empirically unreliable mechanism for preventing mistakes.

[beardbandit]

There is a human in the loop that either prompted the agent or approved the code. So it doesn't matter if the AI is accountable or not.

[the_af]

I hear you, but isn't the human in the loop precisely the one who should be putting their foot down and saying "no, the AI shouldn't be writing the tests to begin with", which would bring us full circle?

[bigstrat2003]

Punishing humans does, in fact, prevent mistakes. Or rather, the threat of punishment causes people to be careful to avoid mistakes, and that prevents mistakes. Sure, this doesn't work 100% of the time, but it does work and has throughout human history. Meanwhile, there's no equivalent paradigm for LLMs.

[anonym29]

Even if you could threaten an LLM with punishment for making mistakes, you might get longer CoTs, but that wouldn't prevent mistakes in LLMs. The lack of accountability isn't the reason that LLMs make mistakes - adding accountability wouldn't change anything.