[BriggyDwiggs42]

Actually you kind of could. If you imagine making a normal hammer slightly more squishy, thats pretty similar to what they’re doing with llms. If the squishy hammer hits a person’s head, it’ll do less damage, but it’s also worse for nails.

[mattigames]

That's quite a big stretch, there are millions of operations where the LLM would do the exact same even if without those "guards", a lot the work for advertisement, emails, and a lot other use cases would be the exact same; so no, the comparison with a squachy hammer is off the mark.

[BriggyDwiggs42]

I remember the result from the sparks of agi paper that fine tuning for safety reduced performance broadly, if mildly, in seemingly unrelated areas

[mattigames]

Fair enough.