[satvikpendem]

> Compounding the problem, labs in China often release dual-use capable models as open-weight. Once a model is open-weight, safeguards that do exist can be removed, making the model available to any state or non-state actor to use for malicious purposes, including the cyber and CBRN misuse those safeguards were built to prevent.

https://www.anthropic.com/research/2028-ai-leadership

[sterlind]

I loathe Anthropic. many companies don't contribute to open-source, but for one to be actively hostile to open-source, to the degree they're lobbying the government to ban it, is uniquely evil. at least these gatekeepers call themselves what they are.

scraping CoT won't stop the advance of Chinese models. neither will a US "ban" on using such models. at this point I'm cheering for DeepSeek or Qwen to catch up to Anthropic. I support anyone who releases open weights.

[xoxolian]

Is OpenAI significantly better so far regarding this, at least publicly? I'm increasing my LLM spend this weekend, and this could impact my decision. And I'll prioritize supporting open-weight models moving forward — already Chatgpt's censorship and surveillance dissuade from asking it genuinely helpful questions.

[sterlind]

OpenAI seems marginally better. they did release gpt-oss-120b, which was decent at the time. but certainly not much better, and they seemed even more on board with fully disabling guardrails for Uncle Sam than Anthropic was. then again, rumor has it that Anthropic's AI selected that Iranian elementary school as part of Palantir's Project Maven pipeline, so..

I strongly recommend open-weight wherever you can. assume any data you pass to a closed model (including opinions or political positions you intimate) will be retained and analyzed in unfriendly ways, either now or ten years from now.

[resident423]

I would say I agree with Anthropic on open source for the reasons stated above like cyber crime, CBRN etc, but I'm interested to hear the other side of the argument. What would be the argument for open source over closed source?

[thriejdiejd48]

The same "open source is too dangerous" argument was used against nmap and other "hacking" tools. The only solution in long term is to fix security issues.

[resident423]

I can understand this for hacking tools, but I'm not really sure how we fix the security issues on the CBRN side? We can't patch the human body like we can with software, so if the model has strong biological capabilities and is released open source, what stops it being used to construct new viruses and things like this?

[n2h4]

anthropic's reasoning is same as "knives kill therefore knives bad".

having open-weight models allows users to use/modify them in novel ways.

[sterlind]

the succinct argument: I don't want arguably the most important invention in human history to be gatekept by a small handful of oligarchs.

I don't trust Dario Amodei, Sam Altman and Elon Musk to act in my best interests. Closed models will have an incredible centralizing effect, and concentrate power like we've never seen since the feudal ages.

If you want to see what it's like for the economy to collapse into a single, extremely valuable commodity, under the control of a small elite, look at Saudi Arabia.

also, I just value freedom tremendously. I want to tinker with model weights. I want to build my own stuff. I don't want to sharecrop in someone's walled garden.

I also worry a great deal that OAI and Anthropic will bow to political pressure and make Claude and ChatGPT push certain political agendas, to report biased information, or refuse to help with legal requests that conflict with corporate values. I also worry about privacy and mass surveillance - chat logs are far more intimate than my search queries or selfies.

[resident423]

I agree with all of these points, my view is just that open source doesn't really do much to prevent it. I also think it adds the additional danger of making dangerous capabilities widely available to anyone, like the ability to design novel viruses which is something that we can't really defend against once it's out there. If anything, putting this kind of capability in the hands of anyone with a GPU could create justification for a mass surveillance state or further concentration of power.

I also just don't think the open source movement has much chance of competing with the city sized data centres owned by Anthropic and OpenAI, or the hundreds of billions of dollars they have available to hire the best researchers. It costs hundreds of millions to train a frontier model, this kind of compute isn't available to the open source community.

[scottyah]

Out of curiosity, what's your stance on gun ownership?

[sterlind]

3D2A. I support repealing the machine gun ban. (and I don't even own a gun.)

[scottyah]

I'm impressed you stick to a pretty absolute devotion to freedom. I get more bitter the older I get, it seems easier to psyop someone into abusing their rights than to get people to fight for and be proper custodians of them.

Especially drugs- I used to think all people should have access, but overall I really wish meth just never existed and people wouldn't distribute it outside of specific circumstances. Being able to cause irreparable damage in one moment of weakness is terrible for people who have less control, and for society as a whole really.

[buu700]

To be fair, those aren't contradictory positions. I'd rather meth not exist, but given that it does exist, I'd prefer to let that revenue go to Big Pharma than North American ISIS.

(That's before even touching the can of worms of allowing the government to criminalize personal health choices, which feels like a glaring loophole in the Constitution to me.)

[iso1631]

> pretty absolute devotion to freedom

Freedom from one perspective

[sterlind]

from which perspective is it not freedom?

[ddxv]

Anthropic wants to ban people using AI. Their moat is going to be using government to gatekeep free and open source AI models.

[smallmancontrov]

> cyber misuse

He who controls the porn controls the universe. - Baron Amodei