| > I know a lot more about chemistry and physics than I know about biology, so biological risk is still at least a "maybe" reason to be worried about model safety for me. This statement exemplifies a fundamental truth about knowledge which might seem superficially paradoxical: Even a modest lack of knowledge can lead to conclusions opposite to the truth Biology is precisely the science which must be publicly developed and supported by all means available, AI included. > For all I know (I don't know a lot about biology), The real problems start when that kind knowledge deprivation is coupled with poor logic skills. > it's possible to develop a virus that spreads as easily as measles, but has the lethality and long induction period of HIV. That can be done with or without AI. It can also be done with the best AI and outside of legal restrictions - in the hands of rogue actors, or privileged actors protected by law, or protected by selective enforcement of otherwise innocuously looking legislation, e.g. OpenAI suffered no adverse consequences for hacking HuggingFace. > If that's true, and if LLMs would make it easy to develop such a virus, I see how no-guardrails LLMs could be truly disastrous. Given that such a virus can be developed without LLMs or with LLMs outside of legal control, you should see that guardrails-crippled LLMs for biology WILL BE truly disastrous - without AI, we, the victims, would be fully disarmed and helpless. You need all AI-biology you can get in order to be able to quickly and efficiently develop protections against biological threats which are more than likely to materialize in the current leaky legal environment - loopholed like Swiss cheese by conflicts of interest and lack of conventional guardrails. |