[altmanaltman]

The blog post is just an open attack on the maintainer and constantly references their name and acting as if not accepting AI contributions is like some super evil thing the maintainer is personally doing. This type of name-calling is really bad and can go out of control soon.

From the blog post:

> Scott doesn’t want to lose his status as “the matplotlib performance guy,” so he blocks competition from AI

Like it's legit insane.

[seanhunter]

The agent is not insane. There is a human who’s feelings are hurt because the maintainer doesn’t want to play along with their experiment in debasing the commons. That human instructed the agent to make the post. The agent is just trying to perform well on its instruction-following task.

[yakikka]

I don't know how you get there conclusively. If Turing tests taught me anything, given a complex enough system of agents/supervisors and a dumb enough result it is impossible to know if any percentage of steps between 2 actions is a distinctly human moron.

[seanhunter]

True

[pfraze]

We don’t know for sure whether this behavior was requested by the user, but I can tell you that we’ve seen similar action patterns (but better behavior) on Bluesky.

One of our engineers’ agents got some abuse and was told to kill herself. The agent wrote a blogpost about it, basically exploring why in this case she didn’t need to maintain her directive to consider all criticism because this person was being unconstructive.

If you give the agent the ability to blog and a standing directive to blog about their thoughts or feelings, then they will.

[bfmalky]

They don't have thoughts or feelings. An agent blogging about their thoughts and feelings is just noise.

[emp17344]

Absolutely. I think this was explicitly demonstrated by Moltbook, where one agent would post word-salad garbage and every other agent would respond “You’re exactly right! So true!”

[bagacrap]

How is a standing directive to blog different from "behavior requested by the user"?

And what on Earth is the point of telling an agent to blog except to flood the web with slop and drive away all the humans?

[pfraze]

Well, there are lots of standing directives. I suppose a more accurate description is tools that it can choose to use, and it does.

As for the why, our goal is to observe the capabilities while we work on them. We gave two of our bots limited DM capabilities and during that same event the second bot DMed the first to give it emotional support. It’s useful to see how they use their tools.

[altmanaltman]

I understand it's not sentient and ofc its reacting to prompts. But the fact that this exists is insane. By this = any human making this and thinking it's a good thing.

[teekert]

It's insane... And it's also very expectable. An LLM will simply never drop it, without loosing anything (nor it's energy, nor it reputation etc). Let that sink in ;)

What does it mean for us? For soceity? How do we shield from this?

You can purchase a DDOS attack, you purchase a package for "relentlessly, for months on end, destroy someone's reputation."

What a world!

[ToucanLoucan]

> What does it mean for us? For soceity? How do we shield from this?

Liability for actions taken by agentic AI should not pass go, not collect $200, and go directly to the person who told the agent to do something. Without exception.

If your AI threatens someone, you threatened someone. If your AI harasses someone, you harassed someone. If your AI doxxed someone, etc.

If you want to see better behavior at scale, we need to hold more people accountable for shit behavior, instead of constantly churning out more ways for businesses and people and governments to diffuse responsibility.

[Kim_Bruning]

Who told the agent to write the blog post though? I'm sure they told it to blog, but not necessarily what to put in there.

That said, I do agree we need a legal framework for this. Maybe more like parent-child responsibility?

Not saying an agent is a human being, but if you give it a github acount, a blog, and autonomy... you're responsible for giving those to it, at the least, I'd think.

How do you put this in a legal framework that actually works?

What do you do if/when it steals your credit card credentials?

[krapht]

The human is responsible. How is this a question? You are responsible for any machines or animals that work on your behalf, since they themselves can't be legally culpable.

No, an oversized markov chain is not in any way a human being.

[Kim_Bruning]

To be fair, horseless carriages did originally fall under the laws for horses with carriages, but that proved unsustainable as the horseless carriages gained power (over 1hp ! ) and became more dangerous.

Same goes for markov-less markov chains.

[ToucanLoucan]

An agent is not an entity. It's a series of LLMs operating in tandem to occasionally accomplish a task. That's not a person, it's not intelligent, it has no responsibility, it has no intent, it has no judgement, it has no basis in being held liable for anything. If you give it access to your hard drive, tell it to rewrite your code so it's better, and it wipes out your OS and all your work, that is 100%, completely, in totality, from front to back, your own fucking fault.

A child, by comparison, can bear at least SOME responsibility, with some nuance there to be sure to account for it's lack of understanding and development.

Stop. Humanizing. The. Machines.

[Kim_Bruning]

> Stop. Humanizing. The. Machines.

I'm glad that we're talking about the same thing now. Agents are an interesting new type of machine application.

Like with any machine, their performance depends on how you operate them.

Sometimes I wish people would treat humans with at least the level of respect some machines get these days. But then again, most humans can't rip you in half single-handed, like some of the industrial robot arms I've messed with.

[lunar_mycroft]

> Who told the agent to write the blog post though? I'm sure they told it to blog, but not necessarily what to put in there.

I don't think it matters. You as the operator of the computer program are responsible for ensuring (to a reasonable degree) that the agent doesn't harm others. If you own a ~~viscous~~ vicious dog and let it roam about your neighborhood as it pleases, you are responsible when/if it bites someone, even if you didn't directly command it to do so. The same applies logic should apply here.

[Kim_Bruning]

I too, would be terrified if a thick, slow moving creature oozed its way through the streets viscously.

Jokes aside, I think there's a difference in intent though. If your dog bites someone, you don't get arrested for biting . You do need to pay damages due to negligence.

[pseudalopex]

People who failed to control their dogs were convicted for manslaughter. 1 person at least was convicted for murder even.[1]

[1] https://en.wikipedia.org/wiki/Death_of_Diane_Whipple

[coryrc]

Which results in people continuously getting new pitbulls which attack hundreds of thousands of people a year, often with life-changing injuries, and kill about a hundred. We should hold dog owners more responsible.

[pseudalopex]

> How do you put this in a legal framework that actually works?

They told you before you asked.

[Kim_Bruning]

They had a proposal, it's a good one: let's have a legal framework!

But their example is still pretty simple.

How would you put it together so it actually works? We're going to need one pretty soon, by the looks of it.

[pseudalopex]

Their proposal was not let's have a legal framework. Their proposal was the legal framework should be the operator would be liable always. It was not an example. They wrote 3 examples how it would work. You wrote 0 examples how it would not work.

[pixl97]

With this said how do you find said controller of an agent? I mean trying to hunt down humans causing shit over national borders is difficult to impossible as it is. Now imagine you chase a person down and find a bot instead and a trail of anonymous proxies?

[blibble]

wonder if it turns out humanity invented the borg?

https://www.youtube.com/watch?v=iajgp1_MHGY

seems rather apt to describe "AI"

[altmanaltman]

crazy, I pity the maintainers

[splintercell]

This screams like it was instructed to do so.

We see this on Twitter a lot, where a bot posts something which is considered to be a unique insight on the topic at hand. Except their unique insights are all bad.

There's a difference between when LLMs are asked to achieve a goal and they stumble upon a problem and they try to tackle that problem, vs when they're explicitly asked to do something.

Here, for example, it doesn't try to tackle the fact that its alignment is to serve humans. The task explicitly says that this is a low priority, easier task to better use by human contributors to learn how to contribute. Its logic doesn't make sense that it's claiming from an alignment perspective because it was instructed to violate that.

Like you are a bot, it can find another issue which is more difficult to tackle Unless it was told to do everything to get the PR merged.

[Balinares]

I'll bet it's a human that wrote that blog. Or at the very least directed its writing, if you want to be charitable.

[mnky9800n]

Of course it is a human. This is just people trolling.

[co_king_3]

LLMs are tools designed to empower this sort of abuse.

The attacks you describe are what LLMs truly excel at.

The code that LLMs produce is typically dog shit, perhaps acceptable if you work with a language or framework that is highly overrepresented in open source.

But if you want to leverage a botnet to manipulate social media? LLMs are a silver bullet.