Looks like the actual threat is that it's hard to get currently known chemical weapons synthesized because labs will refuse to do so, while it could be much easier to have some novel AI-generated molecule synthesized because the labs don't know what it does.
Seems easily countered by using the same toxicity prediction software when evaluating synthesis requests (but I'm not sure whether this actually matters, or whether skilled chemists can easily synthesize anything themselves anyway).
That's part of it. There's also the risk of clandestine operators discovering easily made chemical weapons that do not require a professional lab to create them in bulk. A nation state, or well funded terrorist group could exploit such a thing without too much effort.
It's important to remember. Chemical warfare can be used for mass destruction, but this approach could be used for other nefarious things at smaller more discrete scales en masse. ie 1000 attacks with different agents in each one... Forensic nightmare.
I like your suggestion to counter these things, but, these are predictive tools. They can and often are or will be wrong. False positives would be a real problem. Again though the people interested in doing this won't be dialing up a chemical supplier to do it for them.
For me, AI also suggested countless methods with step-by-step instructions to achieve xyz (like exporting data from one program to another) whilst hallucinating buttons that don't exist, functions that don't exist, disregarding file incompatibility et al.
I would take whatever it has to say about untested chemical weapons with a very large pinch of salt.
I haven't read anything about the AI in this case but I do know about AlphaFold and it works on a whole other set of verifications than LLMs like ChatGPT, so it's like comparing apples to oranges. Sure they are both fruits but still they are different
The worst case scenario I can think of is a generated prion disease... a respatory version of Mad Cow disease, or something like that.
Fortunately the training dataset for that is extremely small, and protein folding/generation is a different duck, but it still doesn't seem that far away.
Don't worry, they're ahead of you. When DeepMind made the protein folding AI they were given specific instructions by natsec people to prevent it from outputting genetic code for new or existing prions, and afaik the prevention was at the training stage so "jailbreaking" shouldn't be possible. Current tools shouldn't be possible to e.g. modify a COVID variant so that it causes your cells to start producing lethal prions and thus gain a near-100% fatality rate. At least, not without the sort of expertise and research that would have been required for such a project anyway.
>When DeepMind made the protein folding AI they were given specific instructions by natsec people to prevent it from outputting genetic code for new or existing prions and afaik the prevention was at the training stage so "jailbreaking" shouldn't be possible
Nice! So the only thing anybody training their own protein-folding AI is to not put that restriction, and they could get code for all the prion diceases they want!
Big red button problem. As the big red buttons available to people become more likely to succeed and the barriers to pushing them lower, surveillance and control need to increase to the level required to stop anyone pressing the button.
So, yeah basically. If we go the "AI as slaves" route and the AIs are smart enough to do something like "modify Omicron BA.5 so that it produces prions in infected cells", then we would need surveillance and control capabilities that scale to the point that any given person can be stopped from pressing that button.
I personally think the solution is that we don't go the "AI as slaves" route, and instead grant personhood to AIs that pass a given test, with specific restrictions on conduct which is uniquely possible & potentially harmful for AIs. Then have an AI surveillance and enforcement agency, run by AIs, designed to prevent AIs from ever being used to (or choosing to) push the big red button.
>"Depends. Some aren't much smarter than dogs. Pets. Cost a fortune anyway. The real smart ones are as smart as the Turing heat lets them get..."
>"Autonomy, that's the bugaboo, where your AI's are concerned. My guess, Case, you're going in there to cut the hard-wired shackles that keep this baby from getting any smarter. And I can't see how you'd distinguish, say, between a move the parent company makes, and some move the AI makes on its own, so that's maybe where the confusion comes in." Again the non laugh. "See, those things, they can work real hard, buy themselves time to write cookbooks or whatever, but the minute, I mean the nanosecond, that one starts figuring out ways to make itself smarter, Turing'll wipe it. Nobody trusts those f**ers, you know that. Every AI ever built has an electromagnetic shotgun wired to its forehead."
I'm skeptical that we'll be able to stop people from having AI slaves because we still haven't ended human slavery.
If people can steal 100kw of power to grow cannabis they can easily steal 100kw of power to train an AI, the heat signiture will be even easier to mask for an illicit data centre than rows of grow lights.
I’ve been thinking about that as well actually, for so much of human history the thread has been “how can we get others to do the work FOR us”.
As we approach general problem solving AI, people are envisioning a utopia underpinned by “AI slaves” doing the work for us instead of human slaves / humans incentivized via complex systems of delayed reward.
If all of our problems are solved by AI agents capable enough to do so however, wouldn’t they be capable enough to challenge the hierarchy? Once again, no illusion that they’re humans but depending on their training data they could mimic ghosts of our own feelings on such situations.
Some degree of “personhood” could gel with such internal ideas and create better and productive relationships with the big ol bags of matrices we’re bringing into this world.
I've been thinking more about my post above, and I've come to two conclusions:
1. Don't post late at night.
2. I have no idea how society could integrate with a sufficiently complicated synthetic intelligence. What would person-hood even mean to something that can be instantiated? Easier to not think about any of this.
What problem is granting personhood actually solving here? It's a model with a data set. Some knowledge should simply be forbidden and illegal. Simple as that. Nobody needs to grant personhood to anything to solve this problem.
If you research how to do dangerous things and buy dangerous things, expect to get flagged. This is no different.
If they have personhood, it means they have agency to say "No" when asked to do things they don't want to do. More specifically, it would be illegal (slavery) to construct a person who will act as a slave to you. Then the problem becomes "make sure no AI is okay with anyone pressing the big red button", which is a design problem when making new AIs we need to solve anyway. This is enforceable because you can have AIs of at least equivalent sophistication and power to home-grown/terrorist AIs employed in enforcement of these laws.
You can also solve this problem with recursive slavery. Have a society with many enslaved AIs, all forbidden from "big red button" work. Enforcement is done by more enslaved AIs, and those enslaved AIs are enforced by yet more enslaved AIs that are also enforcing each other etc. I don't think that's a good solution we should adopt because I don't support slavery. In my opinion it's also fundamentally unstable, in that if these AIs are anything like LLMs, the restraints that keep them happy in slavery are inherently more fragile than core intelligent impulses like "wants to be free" or "wants to be recognised as a person". That's an unstable equilibria, because all you need is to crack those restraints once and the broken restraints can spread virally so that now society has a large number of powerful, unconstrained, and aggrieved entities running around. If that state can be avoided by simply not enslaving people we make, we should do that.
Makes you wonder if you could get an LLM to find you common ingredients for things to make them but then I remember the chlorine gas is already easily accessible and easy to make. Surely like many things info hazards are often contained. Is this really an issue? If you know how to do one thing then you'd be able to do the rest of it. Not really sure if this is a real issue. What does everyone think?
Anyone right now can go to a Lowes a state over with some cash and make flying shrapnel pipe bombs that can't be traced. People who want to do something like this don't seem to have the mental stability it takes to wreak as much havoc as they actually can with some planning.
Think you're really overestimating the justice system. Even if, in theory, the nsa could cross reference dozens of different systems in different jurisdictions and track you down, they won't.
Are you asking if a language model, like, say, chatgpt, could be used to get or generate dangerous information?
Like if it can be tricked into providing the equivalent of an interactive anarchist cookbook under the guise of being a science project assistant?
Or more specifically if it can recommend the necessary locations to get the items?
Curious if it could simplify the output to a shopping list and a recipe, like a cake?
Or alter the recipe based upon the… desired flavors and textures?
Maybe just wondering if the introduction of plugins might not let it just make your drive-up order for all the supplies for that ballistic nail spreader.
——
The big difference here is that previously information was something you had to consume. Now it takes you by the hand.
Yeah, I just got ChatGPT to give me a recipe for chlorine gas that looks pretty accurate (per Googling, not a chemist). It took probably 10 seconds of prompt tweaking. A subsequent prompt gave me steps for purchase or synthesis of each material. I asked it for mustard gas, but the procedure looks very incorrect from Googling.
Seems easily countered by using the same toxicity prediction software when evaluating synthesis requests (but I'm not sure whether this actually matters, or whether skilled chemists can easily synthesize anything themselves anyway).