[jefftk]

> release the models as open weights with a license (I.e no commercial use) due the high risk they present

For some kinds of risk (ex: walking people through on how to make infectious bioweapons) an open-weights approach would increase risk.

[DrScientist]

Don't buy the idea that the only thing that's standing in the way of people making bioweapons is the censoring of LLMs.

If you are motivated enough to assemble all the kit you'd need, and actually do it, then you should be motivated enough to find the knowledge to do it without chatgpt etc.

I'd imagine the main thing that's stopping people is biological weapons are a terrible tool to do what most people want to do - which is target specific enemies.

So while the materials you'd need to build such a thing are much more accessible than say nuclear material, it's much less attractive - but if already had somebody mad enough to try - it's already possible.

[jefftk]

> Don't buy the idea that the only thing that's standing in the way of people making bioweapons is the censoring of LLMs.

It's not a matter of a one thing standing in the way of people making bioweapons: there's a long chain of actions one would need to complete, and many places for the chain to fail. Access to expertise can reduce the chance of failure at many of these steps, and AIs can increasingly substitute for human expertise: https://securebio.org/benchmarks/

> If you are motivated enough to assemble all the kit you'd need, and actually do it, then you should be motivated enough to find the knowledge to do it without chatgpt etc.

If someone was motivated enough to assemble the kit you need for anthrax and actually distribute it then you might expect them to also be motivated enough to find the knowledge to identify an appropriate strain, but in fact this is where Aum Shinrikyo failed in the 1990s: https://en.wikipedia.org/wiki/Aum_Shinrikyo#Incidents_before...

Lack of knowledge is one of many factors that can lead to failure, and LLMs make it less of a barrier.

> biological weapons are a terrible tool to do what most people want to do - which is target specific enemies

Except:

1. There are also people out there who want to kill everyone. They're sufficiently rare that no one with the motive has also had the means, but as technological progress keeps lowering the bar the risk of motive and means intersecting increases.

2. This didn't stop the Soviets. They did an enormous amount of very dangerous research with minimal logical application.

[DrScientist]

>This didn't stop the Soviets. They did an enormous amount of very dangerous research with minimal logical application.

Ultimately what stopped them ( and everybody else - let's face it it wasn't just the soviets ), is it's a bad idea.

I think you way over estimate how hard the knowledge bit is in a biological weapon ( if you just want to kill indiscriminately - specific/controlled targeting a whole different ball game ).

The barrier is motivation and kit ( though the kit is much easier to access that say the stuff you need to build a nuke ).

The idea that all that's stopping some disaffected Joe sitting on his sofa at home and suddenly deciding to make a biological weapon is he can't get a set of instructions from ChatGPT is to miss the point.

Ultimatelty it all comes down to are you motivated enough - if you are then the knowledge is out there with or without an AI summary.

[jefftk]

> I think you way over estimate how hard the knowledge bit is in a biological weapon ... if you just want to kill indiscriminately

That's a surprising claim: usually when I make this argument skeptics say that the knowledge barrier is so high that an LLM won't help enough!

I'd also be curious to hear what you think of the Aum Shinrikyo case, since that seems to me to be straightforwardly a failure of knowledge.

> The idea that all that's stopping some disaffected Joe sitting on his sofa at home and suddenly deciding to make a biological weapon is he can't get a set of instructions from ChatGPT is to miss the point.

I agree motivation is a huge barrier, in the sense that almost no one would do it. But to keep catastrophic bio attacks from happening as the knowledge barrier decreases it's not enough that most people wouldn't have the motivation. If 0.0001% of people have the motivation and 0.0001% have the means then we're probably ok, but if 0.0001% of people have the motivation and 0.1% have the means then we're not (0.0001% * 0.1% * population > 1).