[jbritton]

I’m sick of LLM refusals. I think there are extremely few things they should refuse, like maybe making nuclear weapons or something along those lines. Once you put people in charge of deciding what you shouldn’t be allowed to see that list will grow and grow.

[dgacmu]

Do we really care if an LLM regurgitates information already available in public about the design of nuclear weapons? They're not being trained on restricted material.

(My personal guess is that you don't want them answering questions about some things because you don't want people to try it and blow themselves up, or poison themselves. That's probably much more pertinent to making drugs or conventional bombs, since presumably the average internet user doesn't have a stockpile of HEU sitting around. It's kind of like the reason the Anarchist's Cookbook is a bad idea: using its recipes is likely to be quite hazardous to the cook!)

[handoflixue]

A talented 17 year old can do quite a bit of damage with nuclear materials: https://en.wikipedia.org/wiki/David_Hahn

I'd personally prefer that to be limited to the sort of person who can understand the science, not "anyone with an LLM" - having an "intelligent", "reasoning" assistant who can help you through anything you don't understand does lower the bar quite a lot, and I would prefer there to be a fair amount of friction.

It's not like the material isn't out there - if you want to learn about this stuff, an LLM will happily point you towards Wikipedia and other public sources, it's just not going to walk you through the assembly.

[henry2023]

Sad to read how things unfolded after his experiment.

[handoflixue]

Huh, what sort of refusals are you getting? I basically never run into them unless I'm actively testing.

The primary safety focus these days is biochemical warfare, which I think is a very sensible idea. There's also malware / cyber-security, where I do think it's good having at least some friction.

Refusals on stuff like copyright are mostly just for PR reasons, and I can't blame the companies for responding to legal incentives there.

[jbritton]

I was trying to find a YouTube video I had seen previously. I ended up using Google to find it. There are two bio ethicists promoting the idea that we should make lone star ticks better at spreading alpha-gal and giving everyone meat allergies. So I guess “engineered” + “alpha-gal” is blocked. I find this idea beyond repulsive.

I asked how California guarantees election security and was told it could not answer that question. Upon further questioning it wouldn’t give specifics but it would give generalities, which ultimately turned into an interesting discussion.

[handoflixue]

I am personally quite happy that it is unwilling to assist you in giving everyone meat allergies. That seems blatantly unethical, presumably quite illegal, and correctly categorized as biochemical warfare.

[jbritton]

You misunderstand me. I was simply trying to find the two people who were saying such things.