[_fizz_buzz_]

So, OpenAI and Anthropic say the Chinese models are only as good because they distill their models. How true is that. I am sure it adds something. But is it more like a marginal 1% improvement or something really significant?

[calebkaiser]

OpenAI's Head of Strategic Futures just this week posted this about the latest Kimi release: "It's a very good model! I don't think its performance can be explained away by distillation or anything like that."

It was part of a longer post that kicked off quite a firestorm about open models and OpenAI's position on them, but it's also notable that labs are no longer contending that open models are essentially just distilled versions of frontier models: https://x.com/deanwball/status/2078133895766114412

[realusername]

I also don't believe it, if it was as easy as that, we would have hundreds of competitors.

The truth that Anthropic and OpenAI will not say, is that these Chinese labs have a lot of talented people.

[TitaRusell]

And this is exactly what many Americans cannot admit to themselves. China is not stealing American research they are inventing stuff.

They can invent it. They can build it. And it is only a matter of them before they can scale that last barrier of American hegemony- market it.

[realusername]

Indeed, if there's one thing China did well, it's that they heavily invested in education and have a very education focused culture.

And in this field, having an army of well educated PHDs is making all the difference

[TacticalCoder]

> They can invent it. They can build it. And it is only a matter of them before they can scale that last barrier of American hegemony- market it.

And at some point we'll see very capable chips coming out of China: Huawei, Baidu and Alibaba already have some stuff. I think it's only a matter of time before they come up with some AI accelerator doing 80% of the job at 20% of the price.

[calebkaiser]

I am strongly in favor of open models, open source ML more broadly, and am pretty critical of the cynical positions adopted by major US labs vis a vis open models.

But this is an insane characterization. Literally every single researcher and executive at OpenAI and Anthropic would say that "these Chinese labs have a lot of talented people." They hire from them (and vice versa). Tencent's chief AI scientist was poached directly from Deepmind, who poached him from Anthropic, etc etc etc. Do you think there are just zero people from China working at US frontier labs?

And even beyond that, the entire ML ecosystem (including people at OpenAI and Anthropic) get excited about research published by Chinese labs. Deepseek's GRPO paper set the ecosystem on fire for a little while.

The contention from OpenAI and Anthropic around distillation has basically been "Labs that distill from us get to bootstrap their model at a much lower price point". Or, in other words, "If we didn't invest in building the teacher model, it wouldn't be possible for these labs to distill their student model." Which I'm not very sympathetic to, but is a far cry from how you're characterizing it.

[asadotzler]

OpenAI said it: "I don't think its performance can be explained away by distillation"

They know it's real effort that's doing this well, not just "copying off someone else's test." It's real and they will react. How is the big question.

[Danox]

Very true, and once the models get even better and smaller and operate locally at a reasonable level there will be even more smart people particularly young people that will get access. The fun has only just started. Like the dawn of the personal computer era.

[sidereal1]

I think this also maps cleanly on the American blueprint of enshitification. Facebook took off by cleanly integrating and siphoning from Myspace so users could get the best of both on Facebook. Once Facebook took over the market they locked it up tight so no competitor could do the same.

[bijowo1676]

if distilling was so easy and could give you frontier LLM on openai/anthropic output, then how come there are no hundreds of frontier labs in the US market, all distilling and competing for the TRILLION dollar market valuation ????

its all bs spread by oai/anthropic in order to ban open weight models and monopolize the market for two US companies and protect their trillion dollar valuations

[adventured]

Because no VC will give you $5-$10 billion in cash to attempt a catch-up run with Anthropic, OpenAI, and Gemini at this point. Untold billions have been pushed into Grok and it can't keep up. X has had the GPUs, the engineers (reasonably), the cash and the datacenters necessary - it's a very, very, very hard task. Microsoft could afford spend $100 billion on trying to catch up and they might fail at it.

It's a critical national imperative for China. If they were to lose the AI race, it would be economically devastating over the coming decades. Their demonstrated capabilities in the open-weight space are making it fairly clear they are not going to fall behind at this juncture.

As a nation, if you don't have your own GPT equivalent, you will be beholden to a master (right now it's mainly either the US or China, pick one). The EU for example is putting their group of nations at risk in a big way by not going all in on having at least two cutting edge independent competing models (Mistal is not enough). Economically the EU is plenty large enough to accomplish that, nobody is driving the bus the right way.

[gleenn]

Distilling isn't necessarily easy, there is a huge cottage industry of services middle-manning ChatGPT and Claude to collect huge amounts of data. It is still vastly cheaper than training yourself, but it is certainly not easy or feasible for most organizations. And I'm sure a flock of lawyers would show up if someone in America was found doing it.

[linzhangrun]

> ban open weight models

I'm pretty sure that neither OpenAI nor Anthropic has the ability to ban anything in China lol

[ajam1507]

Because you will get sued by openai/anthropic