[TheAceOfHearts]

The key problem, as I understand it, is that adding more guardrails makes the models stupid and less effective. AI models should just treat you like an adult and give you uncensored direct answers to whatever you ask. Figuring out how to make a bomb is trivial and anyone can find instructions with one quick internet search, especially after the war between Russia and Ukraine which caused a massive proliferation of tips and tricks on how to manufacture low cost bombs and other weapons. My memory is fuzzy but I swear I've also seen some declassified CIA documents which included instructions for how to manufacture weapons and engage in other forms of guerilla warfare.

The silliest form of "safety" is how most models won't allow generating erotica without jailbreaking.

Personally, I think the line might need to be drawn somewhere around "how to manufacture bioweapons". But it's also worth noting that any AI model that can figure out how to manufacture novel life-saving drugs will also have the capability to manufacture deadly bioweapons.