[elliotec]

It should not be an easy choice to trust the latter. Sure, "literal terrorists" exist, but guess what? State terrorism (and state-sponsored terrorism) in the form of massacres, genocide, torture, forced disappearances, etc. is far more deadly and far more prevalent than fringe ideological groups randomly attacking civilians.

These states and elites with the access to the scary weapons ARE the ones doing the damage. The call is coming from inside the house.

I don't understand your point on nuclear proliferation concentrated in the hands of a few states preventing nuclear war or terrorism. Remember, the only two nuclear weapons used in attacks killed a quarter million people, ~90-95% civilians. By one of the few states who had the power. This is the definitional paradigm of state terrorism. And like, now look at the status quo of nuclear treaties and agreements and proliferation. Not exactly a success story.

It's obviously not worth personally formulating a "defense plan" for an AI-enabled bio attack were I personally to have access to SOA weights, but if history is any indication, I feel pretty confident that the likelihood of that happening remains far greater in the closed-weight, elite-state-access-only scenario than the open, democratized, and collectivized one.

[stratos123]

> I don't understand your point on nuclear proliferation concentrated in the hands of a few states preventing nuclear war or terrorism. Remember, the only two nuclear weapons used in attacks killed a quarter million people, ~90-95% civilians.

Or, another way to put it: an extremely powerful military technology the careless usage of which could destroy the human civilization was only used in one war in history and killed less than a million people, and then it was never used again over the next 80 years. I think this is a success story for humanity, and the NPT was an amazing feat of coordination.

> And like, now look at the status quo of nuclear treaties and agreements and proliferation.

I mean, sure, but giving every state (much less every person) the ability to make nuclear weaponry wouldn't make things any better.

I think the reason we think of this differently is because you believe that states are by default... corrupt, maybe, or unreliable, or evil, while individuals are mostly fine. Whereas I think that most people can't be trusted to make correct decisions on topics like "should we use this dangerous technology", while states at least have some decent track record at this.

[elliotec]

Indeed, the core of this misalignment is that we have a foundational difference of opinion on that.

The only caveat I'd push is not that I believe that "individuals are mostly fine," but that as a whole, separate from the state apparatus, I believe humanity collectively has its own best interests in mind and is capable of achieving more good than states or coercive hierarchies can or want to achieve.

Thanks for the civil discourse!