[keeda]

I think we will enter some novel legal territory with cases like this. Intent is a crucial part of the law, and I wonder if we will see "Yes we built this thing, but we had no idea it could do THIS" as a legal defense.

Or, more formally, "these machines have an unprecedented, possibly unlimited, range of capabilities, and we could not have reasonably anticipated this."

There was a thread a few weeks ago (https://news.ycombinator.com/item?id=45922848) about the AI copyright infringement lawsuits where I idly floated this idea. Turns out, in these lawsuits, no matter how you infringed, you're still liable if infringement can be proved. Analogously, in cases with death, even without explicit intent you can still be liable, e.g. if negligence led to the death.

But the intent in these cases is non-existent! And the actions that led to this -- training on vast quantities of data -- are so abstracted from the actual incident that it's hard to make the case for negligence, because negligence requires some reasonable form of anticipation of the outcomes. For instance, it's very clear that these models were not designed to be "rote-learning machines" or "suicide-ideation machines", yet that turned out to be things they do! And who knows what weird failure modes will emerge over time (which makes me a bit sympathetic to the AI doomers' viewpoint.)

So, clearly the questions are going to be all about whether the AI labs took sufficient precautions to anticipate and prevent such outcomes. A smoking gun would be an email or document outlining just such a threat that they dismissed (which may well exist, given what I hear about these labs' "move fast, break people" approach to safety.) But absent that it seems like a reasonable defense.

While that argument may not work for this or other cases, I think it will pop up as these models cause more and more unexpected outcomes, and the courts will have to grapple with it eventually.

[_aavaa_]

Proving even negligence may be hard, but I definitely think they are negligent.

A) This is not the first public incident on people being led down dark and deranged paths by talking with their AI.

B) They record and keep all chat logs, so they had the data to keep an eye out for this even if the AI itself couldn't be stopped in the moment.

[keeda]

And here's where it can get really freaky: what if someone intentionally released a model that led to harmful or illegal outcomes when targeted at specific contexts. Something like "Golden Gate Claude" except it's "Murder-Suicide Claude when the user is Joe Somebody of Springfield, NJ."

Do we have the tools to detect that intentionality from the weights? Or could we see some "intent laundering" of crimes via specialized models? Taken to extremes for the sake of entertainment, I can imagine an "Ocean's 23" movie where the crew orchestrates a heist purely by manipulating employees and patrons of a casino via tampered models...

Interpretability research seems more critical than ever.

[wolttam]

This was not difficult to foresee. People were discussing the mental health risks of these models before OpenAI intentionally tuned it to be sycophantic in the pursuit of capturing users. This can easily be argued as gross negligence against OpenAI.

[keeda]

Hmm I don't know that it was that obvious in advance... I think the earliest recorded case of AI psychosis was that employee Google fired for being convinced LaMDA was sentient:

https://www.theguardian.com/technology/2022/jul/23/google-fi...

At that point, most of us only had experience with older chatbots and didn't know what GenAI chatbots were capable of, so I thought this was clearly a one-off with a gullible and unwell individual.

And then ChatGPT was released. I could see this was something else entirely, and I could see how people could get tricked into thinking it was sentient. However, I still didn't make the connection with how this would interact with other types of psychological issues.

This could be because I was an outside observer; it's likely being at the epicenter of things, these companies had way more early signals that they neglected, which is the sort of evidence I think these lawsuits will surface. To OpenAI's credit, when they realized ChatGPT 4o was overly sycophantic they did roll it back pretty quickly, but I'm pretty sure the speed with which they've been moving they've glossed over a lot of issues.

[tyre]

Oh boy are you going to love learning about felony murder. You can be convicted of felony murder without having:

1. Killed anyone

2. Been in the same location of where the killing took place

3. Known about the crime taking place

John Oliver does an excellent segment on how batshit these laws are, but suffice to say you can absolutely be convicted without intent. https://www.youtube.com/watch?v=Y93ljB7sfco