[resident423]

I would say I agree with Anthropic on open source for the reasons stated above like cyber crime, CBRN etc, but I'm interested to hear the other side of the argument. What would be the argument for open source over closed source?

[thriejdiejd48]

The same "open source is too dangerous" argument was used against nmap and other "hacking" tools. The only solution in long term is to fix security issues.

[resident423]

I can understand this for hacking tools, but I'm not really sure how we fix the security issues on the CBRN side? We can't patch the human body like we can with software, so if the model has strong biological capabilities and is released open source, what stops it being used to construct new viruses and things like this?

[n2h4]

anthropic's reasoning is same as "knives kill therefore knives bad".

having open-weight models allows users to use/modify them in novel ways.

[sterlind]

the succinct argument: I don't want arguably the most important invention in human history to be gatekept by a small handful of oligarchs.

I don't trust Dario Amodei, Sam Altman and Elon Musk to act in my best interests. Closed models will have an incredible centralizing effect, and concentrate power like we've never seen since the feudal ages.

If you want to see what it's like for the economy to collapse into a single, extremely valuable commodity, under the control of a small elite, look at Saudi Arabia.

also, I just value freedom tremendously. I want to tinker with model weights. I want to build my own stuff. I don't want to sharecrop in someone's walled garden.

I also worry a great deal that OAI and Anthropic will bow to political pressure and make Claude and ChatGPT push certain political agendas, to report biased information, or refuse to help with legal requests that conflict with corporate values. I also worry about privacy and mass surveillance - chat logs are far more intimate than my search queries or selfies.

[resident423]

I agree with all of these points, my view is just that open source doesn't really do much to prevent it. I also think it adds the additional danger of making dangerous capabilities widely available to anyone, like the ability to design novel viruses which is something that we can't really defend against once it's out there. If anything, putting this kind of capability in the hands of anyone with a GPU could create justification for a mass surveillance state or further concentration of power.

I also just don't think the open source movement has much chance of competing with the city sized data centres owned by Anthropic and OpenAI, or the hundreds of billions of dollars they have available to hire the best researchers. It costs hundreds of millions to train a frontier model, this kind of compute isn't available to the open source community.