[bluGill]

I don't care how it happened someone should be arrested for illegal intrusion. Agents don't work on their own, someone is responsible. If nobody else the CEO for allowing something unsupervised.

Hugging face also needs someone arrested for not providing security but that is a lesser charge.

[rubyfan]

Being a poorly equipped victim still isn’t a crime thankfully.

It may or may not be a crime and typically the damaged party is pressing the charges. One would argue there is no actual damage here.

[Analemma_]

Despite the common misconceptions from TV, the victim "pressing charges" isn't actually a thing in criminal cases: prosecutors can choose to put someone on trial even if the victim doesn't want that. In practice this is somewhat rare, but it certainly can happen. In my reply to tokioyoyo below I laid out why this is one instance where the government should prosecute even if HuggingFace doesn't want it to.

[famouswaffles]

Criminal Cases of 'hacking' require specific intent. What you're asking is that the prosecution attempt to prove Open AI intended to infiltrate Huggingface maliciously, all while the victim is saying 'no harm no foul'.

No offense but prosecutors have better things to do with their time.

[munk-a]

Gross negligence can stand in for intent. I believe a rather compelling case could be developed on the basis that a system believed to be capable of this was developed and improperly secured.

[rubyfan]

But what’s the crime? The thing got out of its poorly secured cage and caused what damage? Imagine instead of an agent it was a dog or a chimpanzee that got into the neighbor’s yard and the particular neighbor isn’t even pissed about it.

[irthomasthomas]

Aaron Schwartz was being prosecuted and threatened with 35 years in jail for the crime of saving research papers to a thumb drive. What damage did he cause?

[Analemma_]

If intent matters when LLMs get involved, then we can't do anything even if they kill millions of people. Anything LLM-related should involve strict liability.

[famouswaffles]

Get back to me when they kill millions of people then. Like I said, prosecutors have better things to do.

[bluGill]

I don't care about intent. That it happened is unacceptable. Ignorance is not an excuse

[johntb86]

Intent often matters in the law (aside from certain laws with strict liability). You intentionally drive your car into someone you hate and kill them => murder, go straight to jail. You're driving along normally and someone who's chasing their pet cat suddenly runs into traffic and you hit them => no charges. Sometimes you can be charged with negligence for not taking enough care to prevent something, but then you have to deal with the tricky question of how much care is enough.

[munk-a]

If you're driving along normally and then tie a blindfold around your face and hit someone you've taken actions that still expose you to liability. And, in the case that you hit someone who is chasing their pet cat into traffic then you'd better hope you were following every facet of safe driving - being distracted, drunk or on your phone could easily result in you being charged.

[pixl97]

I mean, in theory the FBI could press charges on this as it was an attack involving interstate commerce.

The chances of this are nearly zero if HF doesn't want it, and even if they did OAI has their bread buttered with the administration.

[rubyfan]

But what do you honestly expect would happen? It’s “an accident” with no actual damages.

[sfink]

I agree with the attacker side. The actions of autonomous agents are absolutely the responsibility of the one or more humans that enabled them to take that action. Whether that means someone is arrested, maybe or maybe not, but at least there should be a hefty fine.

I disagree with the defender side. It's not an unreasonable end state, but we're nowhere near there now. It would require holding company employees legally responsible for the security of their services, which means the risk of being employed as a (defensive) security professional is much higher, which means pay needs to be much higher and insurance needs to be available, etc. It's a very different world.

On the weekends, I'm coding up a list management app with a sync server. It's unreleased but exposed to the internet. (This is not hypothetical.) If that server ends up being used as part of an exploit chain, am I legally liable too?

Forget about age verification, now you want to associate every exposed port on the internet with a legally responsible human?

[perching_aix]

> Agents don't work on their own

This is factually false: they both can and clearly did operate in an autonomous and unsupervised manner: https://openai.com/index/hugging-face-model-evaluation-secur...

This does not require sentience, personhood, a soul, or anything of the sort. It further doesn't mean an erasure of legal responsibility, not in principle, and not in historical practice.

I wish people would finally stop with the spiritualistic reasoning around this.

[magicalist]

>> Agents don't work on their own

> This is factually false

From your link:

> After investigating, we now know that this particular incident was driven by a combination of OpenAI models...while being internally tested on a benchmark of cyber capabilities.

Someone set up that test and started it. Whether they outsourced the majority of the work in "setting up" and "starting it" to an LLM or not, they still set it in motion. That's not spiritualistic reasoning.

[perching_aix]

Setting up is not operating.

There's no indication of there having been a human in the loop during its operation: nobody was approving its tool calls, and nobody instructed it to commit these specific actions during its run (via prompting or steering).

There's no indication of any supervision of its operation either: OpenAI's engineers acted with significant delay, long after the agent has already meandered its way through their own infrastructure first.

Given that setting up this contraption in an insufficiently secure manner is almost certainly already a legal liability of equal significance, rejecting this very clear structural distinction is not necessary. That is unless someone is biased towards not wanting to grant the label of autonomy to it, in which case yes, this is absolutely spiritualistic reasoning, hence my point.

I do not want regulation to ride on people's nebulous identification on what specific traits and labels count as human-exclusive. Not just because I deeply disagree that e.g. autonomy would [0], but also because it is entirely unnecessary, for the reasons you also lay out. The agent having operated autonomously doesn't wash OpenAI of responsibility - so why reject the label, if not on a spiritualistic basis?

[0] thousands of years old idea that it is not, by the way: https://en.wikipedia.org/wiki/Automaton -- see also existing regulation recognizing this idea and working with it fine

Edit: one might also want to consider if the law should bite different if there was a human in the loop, or if there were explicit instructions for the agent to take unlawful actions. I'd say yes, and then that also requires this distinction to exist.

[pixl97]

Unlawful action is quite a messy definition. Is performing a vulnerability scan illegal when it's done internally? What about when there's a device on the network that port forwards information to another server you weren't aware of?

[louiereederson]

Well this at least raises the very interesting question of liability for an illegal action, irrespective of whether the action was undertaken by something considered sentient or a person.

[tokioyoyo]

What? Both sides are cool with it, why would anyone be arrested and etc.?

[HumblyTossed]

Because if a human, say a Aaron Swartz type, were to have done it, they'd destroy him.

[arm32]

Hmm, that sounds familiar—like Aaron!

[Analemma_]

Because incentives are aligned properly if agents aren't liability-proof for crimes- making someone go to jail for instances like this is how to get the labs to behave themselves, whereas going "ha ha what an oopsie-woopsie" will make the next instance worse.

Note that for criminal cases (which this was), the justice system can choose to prosecute even if the victim doesn't want that. It often doesn't, but this is one case where it should.

There’s also an optics issue for the justice system at play here: there’s immense public distrust of and anger at the labs right now. I would go to jail if I hacked HuggingFace, even if I said “it was during an eval!”; not doing the same for the labs makes it look like they’re above the law, which is going to make this anger get worse.

[Analemma_]

Actually there's another reason too: because it discourages marketing stunts. Personally I don't think that's what this was, but for the people who do think it was a marketing stunt: you guys should be pushing extra hard for prosecution here. If people can go to jail for AI-alarmism-as-marketing, that makes it a lot less likely to happen again. At the very least, the investigation and discovery process during the prosecution would show us the documents about whether it was a stunt or not, instead of taking OAI's word for it.

[michaelt]

Many legal traditions don’t require a victim. It’s the state that prosecutes, not the victim.

As a practical matter it would be difficult to prosecute an assault where the victim opposed the prosecution, so most states wouldn’t bother - but for things like speeding and dealing drugs the law has been broken despite the lack of a victim.

[dannyw]

I'm imagining a courtroom where the defendant calls upon the victim as a witness who tells the jury to return a verdict of not guilty.

[ethin]

Because a crime was committed, and a pretty serious one at that?

If I blow up your house or steel $100000 from you and we both resolve our differences out of band, should I just be allowed to go about my day like I never did anything, or should I be punished for the crimes I committed? If I am not punished, it makes a mockery of the law that is (supposed) to have protected you, and if it happens repeatedly people will start wondering why the law even should exist if it clearly and obviously doesn't work. Granted, this has yet to happen again, but if OAI isn't punished it sets a very bad baseline precedent: that if I just hack you with an AI model, it's a-okay, and you can't do anything about it because eh, it's all good man!

[ACCount37]

If I steal $100000 from you, and you decide you value your relationship with me more than you value that $100000? Yep, you can just decide to let me have it and let it slide.

Believe it or not, deciding that you weren't wronged and not suing isn't a crime. It happens all the time. What people do with each other is up to them.

Did Bob allow his friend Jack to borrow his truck? No. Does Bob want to sue Jack for taking his truck anyway, and driving it into a ditch? No. Does Jack owe Bob big time for the mess he caused? Yes, but not in any formal legally binding way.

This works for corporations too. When two corporations find themselves at odds, threat of legal action is often used by one company against another as a leverage to resolve things behind closed doors instead. In a more amicable fashion - with no legal expenses of a protracted court battle and no loss of reputation on either side.

[Alpha3031]

A lawsuit is not the same as a criminal prosecution. One is supposed to remedy damages, the other is supposed to be because it is in the public interest that people follow laws and we don't devolve into anarchy (or more dangerously, some might-makes-right society where it is more beneficial to appease the powerful than it is to pursue what is just).

It is possible for a prosecutor to decide it is not in the public interest to persue a prosecution (or that there isn't sufficient evidence to prove a criminal action beyond reasonable doubt), and certainly the victim's opinion could be considered, but ultimately whether criminal charges should be pursued is and ought to be based on a different test to civil matters, one focused on the public interest rather than mere restoration.

[Peacefulz]

I agree with your stance mostly. I get the sentiment, but both of your examples seem civil to me. Both of those situations can and ideally should be resolved out of the courts, and neither carry social weight.

If one entity is injured by another, and subsequently made whole, however the two parties define that, then it is none of my business.

[pixl97]

Corporations commonly hide criminal activity in order to save face.

In the oil industry there is a portion called land management where the portions of oil and gas from wells can be split across a large number of entities. This can lead to numerous complexities that open up opportunities for fraud/theft in division of the profits. Quite often it is easier for the corporation to cover up that this occurred and pay off the person never to talk under NDA about it rather than have to have their customers find out and potentially take millions in losses.

Computer related hacks are very similar. Quite often these are covered up and never disclosed unless the information shows up in public at some point.