[mattigames]

You don't think that if the hammer company had a way (that cost them almost nothing) to make sure that the hammer its never used to attack human beings they wouldn't add such feature? I think many would, if anything by pressure of their local goverment or even the competition ("our hammers can't hurt your baby on accident like those other companies!") , but its impossible to add such feature to hammer; so maybe the lack of such feature its not by choice but a byproduct of its limitations.

[fragmede]

> that cost them almost nothing

Adding guardrails comes at significant expense, and not just financial, either.

[BriggyDwiggs42]

Actually you kind of could. If you imagine making a normal hammer slightly more squishy, thats pretty similar to what they’re doing with llms. If the squishy hammer hits a person’s head, it’ll do less damage, but it’s also worse for nails.

[mattigames]

That's quite a big stretch, there are millions of operations where the LLM would do the exact same even if without those "guards", a lot the work for advertisement, emails, and a lot other use cases would be the exact same; so no, the comparison with a squachy hammer is off the mark.

[BriggyDwiggs42]

I remember the result from the sparks of agi paper that fine tuning for safety reduced performance broadly, if mildly, in seemingly unrelated areas

[mattigames]

Fair enough.