Hacker News new | ask | show | jobs
by simoncion 17 days ago
> If big AI actually make a model that would reliably reject dangerous queries in the first place...

Not happening, for two reasons

1) LLMs mix attacker-controlled data into their instruction stream, and it doesn't look at all like the big LLM providers consider making that impossible to be an actual priority.

2) Intent is what distinguishes "dangerous" queries from prosocial ones. Reliably determining intent is so complex that not even humans can do it.