[tfirst]

Their goal is to downgrade people who are violating their TOS, so I think they'd have some argument there. I have no idea how they'll deal with inevitable false positives, especially given how oversensitive most of the other triggers are.

[dannyw]

The challenge is the examples they’ve mentioned (distributed training infra? ML acceleration techniques?) go beyond what’s prohibited by their ToS and is like a catch net.

I would wager the majority of ML and data science work in the world aren’t frontier LLM development.

[weitendorf]

Yes, this is the problem. They are business interests of Anthropic and have nothing to do with “safety”

[sudoshred]

Safety of their IPO

[Arubis]

This is how I’m going to read all references to AI safety going forward. Brilliant.

[ZetsuBouKyo]

It’s just impossible.

Look at real-life stuff like laws, company policies, or school rules. Humans have to enforce them, and we constantly see crazy cases in the news. There’s no way simple rules can ever make speech completely 'safe.' I can't prove it with math or logic yet, but I have a feeling that it’ll never happen. Even humans can't do it.

We can run a simple thought experiment here. Say Case A violates rule B, so we add rule C. Then Case D violates rule B but follows rule C, so we add an exception... and it just goes on and on like that forever. It never ends. In the end, you just get a massive pile of rules that makes it impossible to get anything done.

Ultimately, we will have to face the truth that knowledge is dangerous.

Giving knowledge directly to people who cannot actually understand it and allowing them to just use it blindly can be extremely unsafe.

To use a real-world analogy, the problem we are facing with weak AI right now is just like the debate over gun legalization. Do we want to risk the abuse of guns or knowledge just to protect the freedom to own them?

[AnthonyMouse]

> I can't prove it with math or logic yet, but I have a feeling that it’ll never happen.

It's not really that hard to actually prove it with math.

It's a computer, so to produce the boolean result (safe or unsafe) there has to be a mathematical formula. This formula will inherently be extremely complex, but even a very simple formula has a huge problem. Suppose "unsafe" is true if X - Y > 0. Make X and Y themselves as simple or complicated as you like but even in the simplest version it's already impossible to calculate unless the model has perfect information.

You can't calculate "X - Y" if you don't know the value of X. And it's indisputable that there is information it doesn't have. Case in point, telling you about a vulnerability in some piece of code is safe (and indeed not telling you is unsafe) if you're the developer and you want to patch it or an administrator and want to mitigate it, but the opposite if you're the attacker and want to exploit it. The model does not know which one you are, therefore it cannot make the correct determination any more than it can solve one equation with two unknowns.

[marcus_holmes]

This is why we have courts and juries. Creating laws that cover all cases and contexts is effectively impossible, so we have humans decide what a fair outcome would be in this specific situation.

[nativeit]

Imagine how many tokens Claude would burn waiting for litigation, not to mention letting it reconsider now that it understands the problem completely!

[AussieWog93]

To make an analogy: Imagine a patron gets banned from ordering alcohol at a particular establishment, because they got too drunk one time.

It's completely reasonable for the establishment to reject a request for an alcoholic drink, and suggest something alcohol-free instead.

It is not reasonable for them to say "sure, here's your alcoholic drink as you requested" and give them an alcohol-free substitute without telling them.

The fact that the patron broke the rules has nothing to do with it.

[prmoustache]

> It is not reasonable for them to say "sure, here's your alcoholic drink as you requested" and give them an alcohol-free substitute without telling them.

Your analogy doesn't work because: - they tell you the rules at the entrance of the bar - they totally tell you when they give you a substitute

The only issue is the bartender asking you for your money before serving you the drink really but again, this is known since day 1 by the customers.

[staticman2]

Your rebuttle seems to be arguing it's okay for a bartender to simultaneously say:

"This is alcohol"

And

"Or maybe it isn't alcohol."

Or to rephrase it, "They tell you the rules at the entrance, they then tell you they don't follow those rules and they are totally serving alcohol even if they are not."

[prmoustache]

No they tell you at the entrance that at any point they may unilaterally decide to replace the alcoholic drink you ordered by a non alcoholic one.

You can decide you are okay with that or not but they aren't dishonest. I wouldn't enter that bar personally but if you do you cannot really complain. It is like complaining because you haven't won at the casino.

[AussieWog93]

I mean, that's not really true either. Nobody is going to read the full terms of service, and they know that.

[loeg]

If it's a violation of ToS, just reject instead of silently downgrading.

[SR2Z]

But then someone would figure out some prompts that don't trigger this, and Anthropic wouldn't be able to try and disadvantage competitors.

[loeg]

Ultimately Anthropic has to compete within the bounds of the law, even when doing an anti-competitive thing would make it easier for them to compete.

[BoorishBears]

Except they openly reject many many other classes of prompts, including extremely high stakes CBRN.

It's only the direction that has direct potential business impact they've decided to sabotage instead of reject.

[vbezhenar]

Their detection is too aggressive. Just today I'm trying to build a kernel for some SBC and I hit that downgrade. I just asked some things about `make menuconfig` items. I suppose it just flags everything related to linux kernel as cyber attacks.

[jchw]

You know, I'm not saying I don't understand what they are doing from a business perspective, but I'm just saying: DeepSeek V4 doesn't silently sabotage you because it thinks you are trying to violate a ToS. Anthropic's clawing back a bit of a moat perhaps, with Fable being an actual improvement of sorts, but now with torching user trust they are really banking on open weight models not catching up to where they are now. I wonder if they have a good reason to believe that they won't, or are hoping for something entirely different to save them.

(P.S. Yes of course I know about model censorship, a different problem, but all of the models are censored to some degree. It happens to be less of a problem for open weight models anyhow, but I figured I'd just preempt this since it's inevitable.)

I actually kinda like DSv4 over Opus 4.7 for some tasks, although I have not figured out what the deciding factor is. (Opus 4.8 so far has not worked very well for me at all, no idea why.)

[literalAardvark]

Anthropic seems to me to have consistently been the baddie despite everyone's posturing.

Not that I expect better from openai but at least they're not pretending to be good.

[thefounder]

They will give you s*t output, that’s how they deal with it. And say that less than 1% of the requests were affected. Think of this like a kind of shadow ban while you still pay top $.

[siva7]

I can't trust any output of Claude anymore as silent sabotage explains many things much better now.

[siva7]

Sabotage is a criminal offense in my jurisdiction, not the legitimate answer to a TOS violation.