[Aurornis]

> The AI can and should refuse it.

This leads to LLMs refusing to do security work for anyone, because it can’t tell if it’s being done for good or evil purposes.

Which is precisely what we’re going through right now with frontier models and it’s terrible.

These proposals always assume some perfect mechanism for identifying the thought crime with triggering on the normal requests. The people who want to commit crimes just persist until they jailbreak the guardrails while the rest of us suffer with denials.

Also if you can’t imagine these guardrails being used by governments to control inconvenient speech, you probably need to think a little harder about the realities of how these will be used.

[throawayonthe]

> Which is precisely what we’re going through right now with frontier models and it’s terrible.

terrible is a bit far, but pretty hilarious for sure

[SiempreViernes]

> This leads to LLMs refusing to do security work for anyone, because it can’t tell if it’s being done for good or evil purposes.

Oh no, of only we had some sort of ideas about automated authentication and authorisation! (If big AI actually make a model that would reliably reject dangerous queries in the first place, they can just use auth tech to serve the current ones to vetted customers)

[simoncion]

> If big AI actually make a model that would reliably reject dangerous queries in the first place...

Not happening, for two reasons

1) LLMs mix attacker-controlled data into their instruction stream, and it doesn't look at all like the big LLM providers consider making that impossible to be an actual priority.

2) Intent is what distinguishes "dangerous" queries from prosocial ones. Reliably determining intent is so complex that not even humans can do it.

[Aurornis]

> they can just use auth tech to serve the current ones to vetted customers

Your solution is to require everyone to surrender their privacy and then allow some company or government decide who is worthy of being allowed to use AI?

It’s crazy how authoritarian, controlling, and anti-privacy the anti-LLM conversation is getting.

[SpicyLemonZest]

I don’t think it’s a privacy issue. All the best auditors collaborate to agree on ethical auditing standards requiring complete disclosure; that’s not a privacy violation, even if it means some businesses end up effectively forced to reveal details they’d prefer to keep private.

[SiempreViernes]

Not sure how "some users get special access after special scrutiny" means "all users must submit to scrutiny", obviously you could just perform the scrutiny upon request.