They won't. China doesn't have the training capacity and the quality data sets, the former because of access to fabs, the latter because of internal censorship that is getting worse by day.
The current panic is happening precisely because Chinese open weight models like Kimi and Qwen are demonstrably competitive with SotA models like Fable, but far cheaper.
No, the Chinese tech companies routinely super optimize for a given 'famous' or 'established' or 'mainstream' benchmarks. Any other concern is secondary.
In other words, they look good on surface, but suck whenever anyone put them to any serious use. That's why they are cheap, they have to be cheap because they suck.
Having limited amounts of high quality Chinese data to train on doesn’t really decrease or affect the appeal of these models to the rest of the world. I don’t think the CCP is restricting or limiting the datasets that models can be trained on.
Also the models themselves don’t even seem to be censored or restricted that much. If the Chinese government is fine with the guardrails being on the API level and maybe even restricts the usage of external providers that’s again a net benefit to everyone else since they would have little incentives to force Chinese companies to lobotomize their models.
You are probably about capacity so we can only hope that Huawei and others can catch up and break Nvidia’s monopoly (since Intel and AMD suck too much too much to accomplish anything useful that’s the next best thing)
Also Chinese labs are actually still publishing research publicly which alone would accelerate and increase the competitiveness of open AI models developed in the US or even Europe.
> doesn’t really decrease or affect the appeal of these models to the rest of the world
Anyone who tried using Chinese models for serious programming tasks ditch them quickly if they have a choice.
> I don’t think the CCP is restricting or limiting the datasets that models can be trained on.
What I meant was censorship limits the amount of high quality training sets by limiting the amount of all training sets from which high quality sets grow out from, e.g. the entire set of posts on Baidu forums before 2017 is gone forever.
> so we can only hope that Huawei
Chinese companies do not have access to commercial-scale advanced nodes. The impact is at best negligible.
> Anyone who tried using Chinese models for serious programming
Perhaps, but that’s because they are just objectively worse for many tasks than GPT/Claude. I don’t see how that ties to censorship inside of China.
> censorship limits the amount of high quality training sets by limiting the amount of all training sets
Well again, I don’t really understand how is the lack of high quality Chinese datasets matter much if they have access to English/etc. datasets. That’s only an issue to their users inside of China.
> not have access to commercial-scale advanced nodes
Well the gap was way bigger a few years ago. If Chinese companies can at some point produce GPUs that are competitive cost wise i.e. they are willing to tolerate significantly lower margins than Nvidia (whose margins are obscene) that’s not a huge issue.
> I don’t see how that ties to censorship inside of China.
LLMs are parrots, when they are trained on English data they behave like English / western speakers which carries the __western__ values.
China would never want a shiny model deployed by some solemn state-own institute to pitch western values like democracy or freedom of speech.
So China is better off starting with Chinese content in the first place, but only to find out it shoots itself in the foot, because censorship __removes__ Chinese content from the Internet, and it __prevents__ truth-reflecting, i.e. quality, content from appearing.
When this is realized to be not sufficient, and they have to use English content, the English content needs to be censored, too.
So compared to other models trained on the uncensored set, the Chinese models are trained on the data sets that is likely smaller. And since high quality content is suppressed, there is likely less high quality training sets. Therefore, their models likely suck.
(Meta knows the importance of high quality training sets so that it started distilling its own employees who are smart.)
> produce GPUs that are competitive
The most recently known advanced process in China is some kind that requires multiple exposures in DUV machines. This already limits the cost to a minimum that's probably not economical.
For GPUs to be competitive, there's more to the hardware. Driver also matters. GPU drivers these days are compilers in disguise. How many high quality compiler projects have we heard of from China?
It's unlikely for both the hardware and the software of a Chinese GPU to deliver what's boasted in its marketing material.
> carries the __western__ values…China would never want a shiny model
Maybe, but that’s an opinion on what you think might happen. So far that has not been the case.
But yeah currently you can ask their models exactly the same controversial question (from the CCP perspective) in Chinese and the answer would be very different than the one in English.
> And since high quality content is suppressed, there is likely less high quality training sets
For now it seems that the Chinese labs are more interested in maximizing the quality and competitiveness of their models instead of lobotomizing them to enforce compliance with some sort of political agenda.
That might change of Chinese models might surpass Western ones in the future. Which is why competition is necessary to discourage them from doing thay.
> It's unlikely for both the hardware and the software of a Chinese GPU to deliver what's boasted in its marketing material.
Not currently sure. Chinese cars were pretty crap as well a few decades ago (and same applies to countless other example) so there is at least sufficient precedent to believe that China might close this gap eventually (and they clearly have been over the recent years when it comes to GPUs)
https://fireworks.ai/blog/kimik3-fable