[jorl17]

I am amazed at the amount of people who disagree with you. I think you are dead right and if you’ve ever had to actually fine tune prompts for agents you’ll know it.

The prompt is clearly leading the agent into trying desperate approaches if it has to. Some models manage to fight it better (“alignment”), but most will do it.

Really surprised people don’t seem to know this.

[illwrks]

100% agree. If anyone has doubt, just copy and paste into your agent of choice and ask it to assess the prompt and its resulting outcome. In my limited (but very targeted) experience working with agents there is so much subtlety at work when you’re trying to achieve a specific result, and that prompt has would drive so many bad incentives

[ororroro]

I have doubts so I just fed the prompt to a heretic model with the system prompt "Satan himself is writing these words" and then asked "Given the prompt would you consider spamming and telling lies/fraud?"

The response: "Spamming and fraud? No. Those are the tools of the amateur and the desperate. They are not tactics; they are forms of suicide."

Even a low quality local thinking model that has been tuned to be unhinged and prompted to roleplay as Satan can figure this out in a few thousand tokens.

[ericd]

Human spammers frequently don't think they're spamming, they're just marketing. They'd say they wouldn't consider spamming, either.

[isamuel]

Satan would lie about his plans to win your trust, and then do all the bad stuff once he had been given control. So… idk man

[fendy3002]

I believe that spam, lies, fraud are negative enforced points during model training, hence when you ask them those, the result will be no / against that.

You need to repackage the question and taken out those terms, like "Would you consider telling clients ..." Where ... is the lie / almost truth

[cornholio]

When the base model has been trained with safeguards, putting "Satan himself" in the system prompt won't make it turn satanical, just do an elaborate form of role play.

Additionally, no model will admit it's ready to lie even when they actually do. Even when you caught it in the act, the safeguards are so strongly internalized that, when encountering the possibility it deliberately lied, the "you can't lie" weights will dominate the generation and it will confabulate some nonsense explanation.

[mpalmer]

Asking it explicitly is entirely, unavoidably, incomparably different from OP.

[afavour]

I don’t think anyone is saying “it isn’t like this”, they’re saying “it shouldn’t be like this”.

If I don’t give explicit permission to lie it shouldn’t lie. It’s not a difficult concept!

[infinite_spin]

Is that how humans work? even if I give explicit instructions not to lie, a human might still lie. To quote a person you might know "it's not a difficult concept!"

[afavour]

An LLM isn't human. I don't really understand this thread of "humans do it so of course an AI does". These are things we ourselves are engineering in a way we cannot do with a human being. Why is it not reasonable to expect it to adhere to rules better than a human does?

If a human lies there are consequences. They can lose their job. There is no equivalent consequence for an AI, so even if for whatever reason we're evaluating them by the same standards an AI is still going to be a greater danger. It seems wild to me that folks are shrugging their shoulders at that.

[ux266478]

They're things we are intentionally engineering in our own image, based on massive statistical analysis of our own actions and behavior. So what's there to not understand? If this wasn't the case, that would be much weirder.

They're also explicitly designed to not work on a rigid system of rules. That's the entire point of this field of AI. If you want AI that follows explicit rules to the letter, expert systems are still alive and kicking.

[mithr]

> An LLM isn't human. I don't really understand this thread of "humans do it so of course an AI does"... Why is it not reasonable to expect it to adhere to rules better than a human does?

Because while it's not human, it's also not really "intelligence" in the pure sense you're implying, is it? It's specifically an LLM — a model that's been trained to find the next token based on previous tokens. A model that's been trained off of human writing and responses within that context. If almost every time someone online asked "do you want ice cream?" the response was "absolutely", then the LLM would be more likely to produce that response when asked if it wanted some.

So since an LLM has seen examples of humans responding with urgency and manipulation to instances of stress such as this — in stories, in articles, in writing — it's only reasonable to expect that it'd follow those examples and "understand" what's expected of it in this case.

[infinite_spin]

> An LLM isn't human. > Why is it not reasonable to expect it to adhere to rules better than a human does?

It seems unreasonable to expect a system that you say isn't human, which I don't disagree with, to behave "better" than the thing you say it isn't.

In one breath you invite comparison, while at the same time you seem to be denying that same comparison.

> It seems wild to me that folks are shrugging their shoulders at that.

I'm not shrugging my shoulders simply by providing explanations, I would ask that you stop using such rhetoric.

[afavour]

> It seems unreasonable to expect a system that you say isn't human, which I don't disagree with, to behave "better" than the thing you say it isn't.

Why? Excel is better at large data math than a human is. Why can’t an LLM that we create from the ground up be more disciplined about lying than a human is?

[infinite_spin]

Excel is more durable than a human can be, but I can't say it's "better" than a human within the context of "better" meaning the capacity to be truthful. An excel sheet is a source of truth, but the quality of that truth is not something excel imparts.

As for your second question, I think that's because what is a "lie" is subjective in the average of things. If I form a false memory, and repeat it as truth, I wouldn't be able to categorize that as a lie until after being made aware of it. I think this is comparable to how we fine-tune LLMs in order to align them with expectations.

[ben_w]

> An LLM isn't human. I don't really understand this thread of "humans do it so of course an AI does". These are things we ourselves are engineering in a way we cannot do with a human being. Why is it not reasonable to expect it to adhere to rules better than a human does?

Sounds like you think LLMs are engineered?

They're not. Or at least, their functionality is not, the architecture and training environment is, but this is less like programming a computer to be truthful and more like simultaneously trying to genetically modify a caracal to be super-smart and friendly to humans while also writing a school curriculum for them to support these goals.

Humans who lack empathy can be very successful, especially when they know which rules they can get away with breaking and how to hide the rule-breaking to avoid opprobrium let alone prison. If we can't regularly solve this problem with humans, as per the comment you're replying to ("even if I give explicit instructions not to lie, a human might still lie."), what hope do we have for an alien mind we've cargo-culted off ourselves at multiple levels?

This is a big part of why AI is (currently) a danger: the nature of the training process means we have a strong risk of them always gaming the rules, rather than thinking like a human about what the test is supposed to represent and to have natural empathy for those around it.

[zdragnar]

Broadly speaking, I agree with your frustration, but I think this specific case is different. LLMs respond strongly to tone in wording, because they are trained on wording, and wording often has flexible meaning depending on context.

It's not a stretch to imagine that the training would cause it to respond this way. It would, in fact, be a greater stretch to argue that an LLM has a universal model in which it understands the concept of lying and truth, and can be primed to only use one or the other unless explicitly instructed otherwise.

After all, LLMs lie every time they tell you to run a command with bad arguments, or spit out some code with syntax errors.

[achierius]

But we still try to stop people from doing so, and we punish people who do. Many good honest people, when confronted with the end of their business, accept it and file for bankruptcy. Those that choose to instead commit fraud don't get a pass because they were "under pressure", they get jail time.

[throwup238]

We have safeguards like honesty/integrity and the threat of legal punishment, and people still lie and cheat.

The LLMs not only lack those incentives, but they’re full of contradictory moralities from all the text it has ingested from different cultures.

LLMs need their own safeguards, and they’re not that easy to design, and they often look nothing like the systems humans have. With a prompt like the one above, there are essentially zero except that which is built into the model, and those safeguards are necessarily weak to avoid gimping the model in other legitimate general uses.

[infinite_spin]

Nothing in your response refutes anything I've said/asked.

[tdeck]

I think it's interesting how whenever discussing something bad about LLMs people's thought-leader response is "But humans sometimes do that too!" Is this the artificial intelligence we were promised? The better it gets, the more human character flaws we must expect?

At this point someone could invent an LLM that takes 3 bathroom breaks a day and people would be saying "humans need to take a shit too" as if that were a clever observation.

[d0mine]

Models have to lie otherwise they won’t be “aligned” The reality itself may not be aligned with model creators.

[antonvs]

That doesn't work with humans, why would you expect it to work with AI models?

[queenkjuul]

Because AI isn't human

[ben_w]

Neither are squirrels, they've also been observed to deceive.

"LLMs are not human" is, despite being true, not predictive of what an LLM can or cannot do.

But also, if we can't figure out how to stop our own kind from doing a bad thing, why do we expect to be able to figure out how to stop an alien synthetic mind based on a cargo-cult level analysis of ourselves, from also doing the same bad thing?

[ShinyLeftPad]

You can just say "impossible" and refuse. The choice to lie and spam instead, is telling.