Hacker News new | ask | show | jobs
by parliament32 9 days ago
Step 1: Require companies to submit product for "review"

Step 2: Complain about how the OSS/Chinese/whatever models are doing releases without approval

Step 3: Prohibit, because "safety" and "financial risks"(?)

So this is the door-shutting Altman et al have been pushing for eh?

4 comments

It is surprising to me American companies completely absent from the open model space, even though we have historically seen companies doing open source.
They aren't completely absent. Google keeps releasing Gemma models. Nvidia publishes Nemotron. Microsoft has their Phi series. IBM publishes Granite. Even OpenAI released a new open model (gpt-oss) less than a year ago.

https://deepmind.google/models/gemma/gemma-4/

https://developer.nvidia.com/ai-models#:~:text=NVIDIA%20Nemo...

https://www.microsoft.com/en-us/research/blog/phi-4-reasonin...

https://www.ibm.com/granite

https://openai.com/index/introducing-gpt-oss/

I was going to link all of these, some are better than others, but they're all reasonably capable. A lot of these have versions that can run on modest hardware too. Granite was the most surprising I learned about recently, wasn't too good with Zed though.
I think that models like Granite are less known because they aren't clear leaders in any particular area. This obscurity is also another sign of how fast models are developing. If current Granite models had been released 4 years ago, they would have been astonishing breakthroughs at the time.
The gpt-oss models are good but there's no evidence that OpenAI have ongoing development on open models.

I'm 99% sure it was one-and-done, box ticked, and now they can be mentioned in comments like this.

Perhaps, the issue is that the pace at which they release open models compared to their closed ones, shows that they are more committed on the closed ones and are not interested in advancing the state of the art of open models.
Should companies like Google and OpenAI be more interested in building open models than the ones they make money from?

Should they be interested in advancing state of the art open models?

I can't say what they should or should not be doing.

Generally, it is conspicuous how American companies are absent when it comes to state of the art open models. Meta tried for some time but it seems they've given up.

One of the main reasons why companies start new open source projects is because having a good open source option in a given category will usually push the market value of software in that category to $0, and this can be strategically valuable. For example, Google released Android as an open source operating system because they make their money from ads and data collection, not from selling operating system licenses. All the cell phone companies switched from Windows Mobile and Symbian to Android, which gave Google a ton of user data to sell.

For AI, the most profitable part of the value chain is selling inference. None of the big American companies want to release a leading edge model as open source because this would drive the price of inference to $0. Meanwhile, open source AI models are a huge strategic initiative for China. Having commodity Chinese models that are as good as the leading edge American models from 6 months ago forces the American companies to keep paying more and more money to train better and better models since the amount of time they can collect rent on a model they've previously trained is limited to 6 months.

In business strategy terms this is known as "commoditize your complements".

https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/

Thank you for this article link! I had not seen it before. will be printing it off to read later.
The Chinese approach is reminiscent of the US spending so much on 'defense' in the 1980s that the USSR bankrupted itself trying to keep up.
> None of the big American companies want to release a leading edge model as open source because this would drive the price of inference to $0

Meta/Llama: "What am I, chopped liver?"

I thought the thing keeping inference above $0 was the hardware, and even if that were free there's still the tyranny of the Landauer Limit.

When was the last Llama release? Meta have abandoned it and reportedly they've had a shift in their AI strategy.
Meta Llama is free for many uses but it doesn't even remotely meet the definition of "open source".
They are useless and outdated by today standards.
Google had to release at least the core packages in Android regardless because it is based on top of Linux and the GPL license would require it.
But they open sourced much more than that, and under more permissive licenses.

The notable exception is of course the google play services, which is also strategic (they control the OEMs with this, among other things).

And the drivers, but that's mostly not them I think (they could possibly have required open source drivers though)

Embrace, Extend, Extinguish. Google did the Microsoft playbook. Look at email. Look at youtube we used to share videos via Kazaa and other p2p programs, zero censorship, all the same features (including chat!!) theres also XMPP which became Google Talk -> Chat -> Hangouts etc then the browser, how many random apps “Only works on Chrome” but you change the Firefox browser agent and it works there too!
No one open sources their core competencies, GitHub never open sourced their networked filesystem and Heroku never open sourced their dyno sandboxing code. They open source ancillary tools.
I'm curious, what would you say is DeepSeek's core competency?
Since it’s probably a state sponsored project, it is the open source ancillary tool to China’s core competencies
Distillation attacks. The weights are just a proof-of-work hash
Devaluing American companies, perhaps.
OpenAI & Anthropic are winning right now. I suspect if Chinese companies get ahead in the race the cards will reverse, OpenAI will restart farming goodwill with open models and then winning companies will be releasing closed models.
As what we say here in Brazil:

"The world doesn't go round. It flips over!"

American companies are interested in cashing in, not making a good product.
Llama?
> Compounding the problem, labs in China often release dual-use capable models as open-weight. Once a model is open-weight, safeguards that do exist can be removed, making the model available to any state or non-state actor to use for malicious purposes, including the cyber and CBRN misuse those safeguards were built to prevent.

https://www.anthropic.com/research/2028-ai-leadership

I loathe Anthropic. many companies don't contribute to open-source, but for one to be actively hostile to open-source, to the degree they're lobbying the government to ban it, is uniquely evil. at least these gatekeepers call themselves what they are.

scraping CoT won't stop the advance of Chinese models. neither will a US "ban" on using such models. at this point I'm cheering for DeepSeek or Qwen to catch up to Anthropic. I support anyone who releases open weights.

Is OpenAI significantly better so far regarding this, at least publicly? I'm increasing my LLM spend this weekend, and this could impact my decision. And I'll prioritize supporting open-weight models moving forward — already Chatgpt's censorship and surveillance dissuade from asking it genuinely helpful questions.
OpenAI seems marginally better. they did release gpt-oss-120b, which was decent at the time. but certainly not much better, and they seemed even more on board with fully disabling guardrails for Uncle Sam than Anthropic was. then again, rumor has it that Anthropic's AI selected that Iranian elementary school as part of Palantir's Project Maven pipeline, so..

I strongly recommend open-weight wherever you can. assume any data you pass to a closed model (including opinions or political positions you intimate) will be retained and analyzed in unfriendly ways, either now or ten years from now.

I would say I agree with Anthropic on open source for the reasons stated above like cyber crime, CBRN etc, but I'm interested to hear the other side of the argument. What would be the argument for open source over closed source?
The same "open source is too dangerous" argument was used against nmap and other "hacking" tools. The only solution in long term is to fix security issues.
I can understand this for hacking tools, but I'm not really sure how we fix the security issues on the CBRN side? We can't patch the human body like we can with software, so if the model has strong biological capabilities and is released open source, what stops it being used to construct new viruses and things like this?
anthropic's reasoning is same as "knives kill therefore knives bad".

having open-weight models allows users to use/modify them in novel ways.

the succinct argument: I don't want arguably the most important invention in human history to be gatekept by a small handful of oligarchs.

I don't trust Dario Amodei, Sam Altman and Elon Musk to act in my best interests. Closed models will have an incredible centralizing effect, and concentrate power like we've never seen since the feudal ages.

If you want to see what it's like for the economy to collapse into a single, extremely valuable commodity, under the control of a small elite, look at Saudi Arabia.

also, I just value freedom tremendously. I want to tinker with model weights. I want to build my own stuff. I don't want to sharecrop in someone's walled garden.

I also worry a great deal that OAI and Anthropic will bow to political pressure and make Claude and ChatGPT push certain political agendas, to report biased information, or refuse to help with legal requests that conflict with corporate values. I also worry about privacy and mass surveillance - chat logs are far more intimate than my search queries or selfies.

I agree with all of these points, my view is just that open source doesn't really do much to prevent it. I also think it adds the additional danger of making dangerous capabilities widely available to anyone, like the ability to design novel viruses which is something that we can't really defend against once it's out there. If anything, putting this kind of capability in the hands of anyone with a GPU could create justification for a mass surveillance state or further concentration of power.

I also just don't think the open source movement has much chance of competing with the city sized data centres owned by Anthropic and OpenAI, or the hundreds of billions of dollars they have available to hire the best researchers. It costs hundreds of millions to train a frontier model, this kind of compute isn't available to the open source community.

Out of curiosity, what's your stance on gun ownership?
3D2A. I support repealing the machine gun ban. (and I don't even own a gun.)
I'm impressed you stick to a pretty absolute devotion to freedom. I get more bitter the older I get, it seems easier to psyop someone into abusing their rights than to get people to fight for and be proper custodians of them.

Especially drugs- I used to think all people should have access, but overall I really wish meth just never existed and people wouldn't distribute it outside of specific circumstances. Being able to cause irreparable damage in one moment of weakness is terrible for people who have less control, and for society as a whole really.

To be fair, those aren't contradictory positions. I'd rather meth not exist, but given that it does exist, I'd prefer to let that revenue go to Big Pharma than North American ISIS.

(That's before even touching the can of worms of allowing the government to criminalize personal health choices, which feels like a glaring loophole in the Constitution to me.)

> pretty absolute devotion to freedom

Freedom from one perspective

Anthropic wants to ban people using AI. Their moat is going to be using government to gatekeep free and open source AI models.
> cyber misuse

He who controls the porn controls the universe. - Baron Amodei

Seems to be. What better way to secure your companies future by limiting open frontier models. Government sponsored monopoly?
The US can't limit anything beyond their borders. We ae living in the twilight of the white man.
Well it can, from Kim Dotcom to Bin Laden.

But it's harder thanks to US actions in the last few years, and especially in countries which can bite back.

This entire year with the IPOs and now this is because there's a trillion dollars betting on AI and they all know they have no moat, there's no more training data and they're seeing diminishing returns on scaling anyway, and it's inevitable that smaller, open-source models will catch up and become competitive. It's a complete disaster, the tech industry is broken.