[versteegen]

Thank you for taking this seriously enough to write this, and anyone else likewise.

But this is attacking a strawman, amateur bioterrorists. AI is a force multiplier in the hands of an expert. If it took a team before, maybe a single malicious actor can accomplish it now that AI can fill in the parts they aren't well-versed in. And that dramatically increases the chance of it happening.

Being at risk of killing yourself also just makes success X% less likely, but if X < 90 that doesn't mitigate much.

[red_green_yell]

It sounds like your claim is "AI is a force multiplier, therefore it is dangerous and must be regulated." Not trying to strawman - I read your comment twice and I think that's what you're saying.

By that logic Excel, the internet, calculators, and air conditioning are all force multipliers in that all of the make it easier for an expert who can make bioweapons able to do so more quickly / comfortably / easily. Obviously it wouldn't make sense to "pace" any of these technologies. Even social media or email could be considered force multiplier in that they could literally allow you to increase the number of collaborators working on supposed bioweapon. Credit cards and modern logistics force multiply your ability to purchase precursors.

Also it would be a force multiplier on the prevention/mitigation side as well. Police armed with AI to scan for suspicious precursor purchases or review security camera footage after the fact would be far more effective than before. It's not obvious that the force multiplication is greater to the bad actor or to those who oppose them.

[versteegen]

I didn't argue "must be regulated". I'm arguing AI is powerful (at achieving things, and hence has dual-use dangers). Many people won't even admit that, which is the part that really annoys me: they don't even want to have a conversation about risks because somehow AI is not actually a powerful general-purpose tool. Of course everything you said is true, though many of those things aren't very powerful. But the internet and social media and smartphones certainly have been of great utility for terrorism and child exploitation (and probably also causing many would-be-terrorists to get picked up).

[areoform]

    But this seems like it's attacking a strawman. AI is a force multiplier in the hands of an expert. If it took a team before, maybe a single malicious actor can accomplish it now that AI can fill in the parts they aren't well-versed in. And that dramatically increases the chance of it happening.

I'm glad that you brought that up, genuinely so.

I want to ask you a question, do you feel like that this is the problem being solved by the current "safety features?"

Let me rephrase my question, do Anthropic and OpenAI implement the same "guardrails" and "safety features" for the NSA? Does the Mythos NSA Edition™ have these restrictions? What about the one that's running as a part of Maven?

How does stopping me from asking about rabbit sex protect you from nut jobs with a security clearance using these systems to go off and make weapons?

Because this has happened before. The only successful bioweapons attack in US history was done by a guy who worked at Fort Detrick. https://en.wikipedia.org/wiki/2001_anthrax_attacks

The only private organization successful enough to make biological and chemical weapons is Aum Shinrikyo and they had university affiliation and nearly a billion dollars in the 1990s.

The guy who led Aum's bioweapons program was

    Seiichi Endo was born in 1961 and attended Obihiro university of Agricultural and Veterinary Medicine. After graduating, he became a student at the Kyoto university medical school research department focusing on AIDS-related gene research at the viral research center. He joined Aum in 1987, became a monk in 1988 and later led Aum’s biological weapons program.

and,

    Among some of Japan's "best and brightest" who joined the cult included a former researcher of the National Space Development Agency of Japan, an expert on chemical weapons who majored in organic physics at Tsukuba University, a researcher who studied elementary particles, a reporter with a major Japanese newspaper, a physicist from Osaka University, a cardiac specialist, and an organic chemist, to name a few.

https://irp.fas.org/congress/1995_rpt/aum/part04.htm

and,

    Through its network, Aum Shinrikyo successfully recruited over 300 scientists and engineers—including employees at the Kurchatov Institute, the premier nuclear facility in Russia—who were either attracted to the apocalyptic ideology or lured with financial incentives.50 Aum Shinrikyo’s Russian contacts enabled group members to access black market materials and hardware.

and,

    In response, the cult constructed the $30 million USD Satyan 7 facility which was equipped with three laboratories, a computer control center, and five reactors, all of which was made from corrosion-resistant Hastelloy, ideal for the production of chemical weapons

Why didn't anyone ask questions about a known cult importing an absurd amount of precursors, industrial-grade HEPA filters, fermenters, and lab equipment?

It's because they'd recruited members of the military, bribed local policemen and politicians, and used their sway to silence critics (or kill them).

How does stopping me from asking Claude about rabbit sex stop people like them?

From where I'm standing, Anthropic and OpenAI would have sold Aum a subscription.

The Fort Detrick guy would have access via the US Government and Aum had university affiliation. I am yet to read a serious proposal that actually deals with these risks and the other real risks of this technology.

[versteegen]

Wow, remarkable. Clearly Aum is a different league from the lone-wolf "Fort Detrick guy", treat the risks separately. But I'll take these questions as rhetorical. (See my reply to the sibling comment.) I can't answer them and I'm not defending the guardrails on Claude; in their current form I too find them pretty ridiculous.

[azeirah]

Please write up your comments as blog posts, articles or whatever you can do.

There is such an incredible amount of bullshit going on in the conversation. You are contributing a point that is necessary for people to start taking into consideration and the only way to get people to talk about it is by repeating it 100 times.

I fear that the AI labs will get their way and we'll get idiotic restrictions that favor them, all in the name of "security" when the reality is they care about power for themselves, not real world security.