[UqWBcuFx6NV4r]

“The only reason that China is good is because it steals from us” spoken from atop the unambiguously crumbling has-been empire.

I’m not American nor Chinese, and I’m from a much more US-aligned country. But, Christ, you lot really are asking for it.

[andy_ppp]

I don’t think you can steal someone’s work if it’s based entirely off theft tbh, I see distillation as smart - are you denying it’s a huge part of why Chinese models are competitive or do you believe the narrative yourself that they are doing this with several orders of magnitude less compute and the US labs are profligate and fond of burning money rather than optimising?

The truth is probably Anthropic/OpenAI/Google are pretty efficient but less efficient than the Chinese labs, the Chinese labs probably have more compute than they say to undermine US spending and distillation is quite efficient at bridging the gap in compute.

[HarHarVeryFunny]

The way you make an LLM smarter is by training it on more data, which means you need to make it bigger, which means it costs more to train.

High quality data is expensive. Synthetic data will get you so far, but after that you need to start paying experts to create data for you, which has been going on for a long time. The latest thing is paying for human written LLM-as-judge AI-output evaluation "rubrics", trying to extend RLVR into areas where "looks like it checks the boxes" is the best you can do.

When anyone, Chinese or not (Elon Musk cheerfully admits to distilling OpenAI models) uses the output of someone else's model to train their own, then what they are primarily getting is cheap training data, but you still need to train your model on this data! You may have reduced the cost/speed of training data acquisition, but if you are training a 3T param model (Kimi 3) then you still need the compute to do that - that did not change.

There was an interesting mention of the cost of training data in the recently leaked DeepSeek investor meeting, where their CEO referred to the cost of human-generated training data in China (i.e. using Chinese labor) as being the same as that in the US, which seems surprising. He also mentioned the time such data takes to be created. No doubt the Chinese will catch up in this area - this is just time and money, not Dutch technology (ASML) that the US is blocking them from buying.

[hereme888]

Phew, if that's not clearly anti-American hate, insult towards Christianity, hatred in general.... Then idk what is.

And btw, yes it's an objective fact from every technical angle that the Chinese only have competitive models because of US tech. Why do you think they try so hard to smuggle NVIDIA GPUs, and now exposed infrastructure to create cheap imitations of American tech?

[HarHarVeryFunny]

Just in case you are interested in any facts ...

The Trump administration, having first blocked China from buying NVIDIA H100's, has since done a U-turn and is now allowing them to buy the more powerful NVIDIA H200, on a case-by-case basis.

Now, the CHINESE government is blocking Chinese companies from buying these H200s, at least in part because it turns out that being denied US tech has been a great accelerator for Chinese tech, with Huawei now producing the entirely domestic Ascend 950 chip, which according to NVIDIA's Jensen Huang performs about the same as NVIDIA's own H100 (which while not NVIDIA's most powerful is still plenty capable, and is what Elon Musk's Colossus-1 data center mostly uses, currently being rented out to Anthropic).

[hereme888]

H100 export licensing restrictions began on August 26, 2022, under the Biden administration.

The CCP tried to promote their own chips, but later regressed and started allowing ByteDance, Alibaba, and Tencent to purchase more than 400,000 H200s. And they are trying to acquire millions.

Thus, export restrictions accelerated Chinese substitution, but also made that substitution slower, costlier, less scalable, and technically inferior to unrestricted Nvidia access.

> According to Jensen Huang, Ascend 950 performs about the same as H100

Source?

[bigyabai]

It's been great. I hope they keep importing them, Qwen and Z.ai have done far more for humanity with that compute than Anthropic or OpenAI ever did.