[whack]

The most compelling argument against Open Source AI: it is analogous to "open sourcing" nuclear weapon technology. Or an how-to guide for making meth/bombs/bioweapons. There is a high risk that we will one day achieve ASI or something close to it. When that day nears, if even one open-source AI user handles their AI in an irresponsible way, we are all FUBARed. In the same way that one irresponsible person with a nuclear bomb can FUBAR the entire world, hence why "open sourcing nuclear weapon tech" is not even a subject of debate.

You don't have to believe that ASI is 100% likely. People like Hinton believe there's a 50% chance of existential risk from AI, but you don't have to go that far either. If there's even a tiny chance of ASI posing a threat to humanity, that's reason enough to lock it down

[danny_codes]

The difference is that people building atomic weapons knew what they were building. OpenAI/Anthropic have no idea how emergent intelligence works. They're just running in the dark towards the cliff. Leaving it in their hands is the same risk as leaving it to any random guy, because random guy and the labs have the same understanding of what they're building. I'm tired of us treating labs like they're somehow uniquely good at AI. They are not. They simply have enough capital to train. If training was affordable they would have no edge whatsoever.

[whack]

I'm not claiming that OpenAI is worthy of trust. The safest course of action is to either stop all AI research immediately, or have it be highly regulated by an international body. The most dangerous course of action is to give every single individual the ability to create their own variant of an ASI

[potsandpans]

This argumentation is going to be joked about in the future. Like downloading a car.

Would you open source a nuclear bomb???