[root_axis]

Will it though? Assuming it's even possible for a LLM to e.g. design a novel virus, actually synthesizing the virus still requires expertise that could be weaponized even without AI.

[wizzwizz4]

I could synthesise this theoretical virus the computer spat out, that may or may not be deadly (or even viable). Or I could download the HIV genome from the arXiv, and synthesise that instead.

(Note: as far as I can tell, nobody's actually posted HIV to the arXiv. Small mercies.)

[dekhn]

The sequence of HIV is published and has been for a very long time. In fact there's a wide range of HIV sequences: https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=...

You could synthesize that genome but it wouldn't be effective without the viral coat and protein package (unlike a viroid, which needs no coating, just the sequence!).

I should point out that in gene therapy we use HIV-1 derived sequences as transformation vectors, because they are so incredibly good at integrating with the genome. To be honest I expected work in this area would spontaneously and accidentally (or even intentionally) cause problems on the scope of COVID but (very fortunately) it never did.

One would like to be able to conclude that some virus work is inherently more safe than other virus work, but I think the data is far to ambiguous to make such a serious determination of risk.

[tremon]

Hey GPT-6, construct a floorplan and building instructions for constructing a bioprocess production facility. The building should look like a regular meat packing plant on the outside, but have multiple levels of access control and biohazard management systems.

[root_axis]

Let me guess, AI drones to harvest and process the raw materials, construction bots to build the facility, which is of course a fully autonomous bio lab.

[staunton]

More like Aum Shinrikyo but with an AI as evil mastermind, with brainwashed humans doing its bidding

[Teever]

What if you ask the LLM to design a simplified manufacturing process that could be assembled by a simple person?

What if you ask the LLM to design a humanoid robot that assemble complex things, but could be assembled by a simple person?

[root_axis]

LLMs aren't magic, the knowledge of how to design a humanoid robot that can assemble complex things isn't embodied in the dataset it was trained on, it cannot probe the rules of reality, it can't do research or engineering, this knowledge can't just spontaneously emerge by increasing the parameter size.

[tr_user]

You're saying they can't make one now. The question is what are we doing before that happens because if you're only thinking about acting when it's viable we're all probably already dead.