[fc417fc802]

I believe a sufficiently advanced model could provide a layman with actionable step by step instructions for building a nuclear weapon. They're complicated but not (AFAIK) that complicated. The more or less insurmountable barrier there is weapons grade material. Thankfully refinement is prohibitive in cost, expertise, and equipment.

In comparison, basic munitions are incredibly simple given a recipe and shop tooling. But just because something is conceptually simple doesn't mean it's a good idea to go out of the way to disseminate step by step instructions.

[BizarroLand]

The difficulty with a fission bomb is getting enough uranium or plutonium or other fissile material together for the bomb yield you want (at least above the critical mass for your chosen material), and refining it to fissile form, (since most fissile material found in nature is a more stable variety), and then separating the fissile bits with something thin but neutron absorptive.

The rest is just slamming the material together with a small explosive so that it passes the critical mass state and starts a chain reaction.

This is information you can find in many places if you're willing to put the effort in to go searching for it. Knowing this knowledge does not get you any closer to making atomic bombs. The process of mining uranium or plutonium is difficult, expensive, and very likely to get you caught before you even make it to the enrichment step of the process thanks to constant world-wide spy satellite surveillance.

Unless you are a nation, your only chance of making a nuclear bomb would be to find a lost nuclear submarine and convert the nuclear material inside of it before you were caught.

[lazide]

A gun type maybe. But then, two paragraphs and some machining knowledge + shop tooling could do the same, given enough refined material.

Ain’t no way a layman is pulling off an implosion device, regardless of tooling or LLM guidance. The explosive lense structure and timing required is quite complex, and would require some significant calculation from someone who actually knew what they were doing.

Nation state, or even sufficiently motivated big corp, if they had the refined material? Sure. Layman? No.

Thinking they can with LLM slop involved? That will make for some very interesting radiological incidents though!

[jerf]

"A gun type" of nuke is sufficient to achieve most, and usually all, of the goals some small group building a nuke would have.

We are all fortunate that as fc417fc802 mentioned, refining the materials proves to be quite challenging and I see no particular way that AI could possibly make that any easier. If it was as simple as building a gun-type nuke banging together any uranium together to get a big bang we'd be living in a very different world.

[fc417fc802]

I agree, but really feel like you're missing the point here. Many things are reasonably straightforward and require almost no understanding when you have simple step by step instructions. LLMs are capable of providing such instructions and in certain cases they probably shouldn't.

But it's not as simple as just refusing help on a broad swathe of topics they way they do now. That makes agents much less useful in general (ie lots of collateral damage) and for many topics is entirely ineffective given that for better or worse the internet already makes such material readily available. In such cases reporting suspicious behavior is likely to be much more effective than denial.

Aside: You've now got me curious and I really want to test the frontier models to see to what extent they're capable of providing sensible designs and specifications for implosion type thermonuclear weapons but also feel like that would attract the wrong sort of attention and probably create a headache for me in more ways than one.

[lazide]

I think you’re missing the point?

The data is often wrong enough it screws whoever tries it unless they have enough experience/knowledge to not need it, or really doesn’t help beyond what someone using existing tools to get - albeit with a little more motivation.

At best, it either gets someone started with something they still need to think to finish, or gets them deep into a mess it can’t help them get out of. In my experience.

In some edge cases, it can be used by experts to automate some grunt work or do prototypes without getting in the way, but often a better thought out framework is usually faster in my experience.

Awhile ago I made an analogy about WYSIWYG gui tools, and the more this comes up, the more accurate I think it really is.

[fc417fc802]

Does that not depend entirely on the topic and does it not get better with each generation? This is a general ethical and functional question that isn't going away about how the models ought to handle certain topics. Much of the difficulty at present is caused by a ham fisted broad censorship approach that I'm pointing out is wrong headed in an at least somewhat nuanced way.

[lazide]

Maybe? I haven’t seen it crop up however on any topic someone knows well - a kind of dunning Kruger, I guess?

And yeah, the censorship model is wrong, but also the underlying other model is wrong too.