[ralfd]

Closed Source AI at least can be controlled. See the directive of the US government regarding Fable (even if one disagrees about the directive there is no doubt that it is effective in shutting it off) or the safe guards by a corporate structure (even a profit driven one). It is schizophrenic to praise Anthropic for refusing the Department of War full access to their models but at the same time root for Open Source models.

Edit: relevant Scott Alexander article from today

https://www.astralcodexten.com/p/my-ai-opinions

> In terms of bioweapons, I expect that closed-source AIs will be heavily optimized against helping with these, and open-source AI will be banned after the first warning shot (or become economically prohibitive even before then).

[zozbot234]

Note that warning shot in that blog post means specifically a near-disaster event (perhaps one that's just barely averted) that's specifically caused by the AI. So far we've had AIUI no significant indication of open-weight AIs being problematic in that sense, whereas one can quibble that proprietary AIs have done dumb and dangerous things.

(For example, I suspect that plenty of folks would view the recently threatened mass scan of the DN42 hobby network as an instance of misaligned agentic behavior that would have wasted non-trivial resources, and I also think that most observers would pin the specific behavior of that AI on a proprietary SOTA model, not an open one. That's clearly not a disaster-level event, but it should scare you if you're concerned about alignment.)