[parineum]

For me, I don't really care about the theft aspect but when people are claiming that these open models are better value or going to overtake anthropic/openai models, the implication that the open models are training of distilled data means all the "progress" they are making is just mimiced from the closed models.

It's a bit interesting how the open models are able to keep pace with the closed models except whole maintaining a steady following time.

[abdullahkhalids]

It's important to note that even if these open models are distilled, they are showing genuine improvements in their architecture, which enables inference costs to be several factors below what equivalent closed models have.

The interesting question is: will Anthropic release a Fable like model with an architecture similar to Kimi, and get the inference cost gains? They should surely beat Kimi because they can internally distill as much as they want.

[orbital-decay]

Distillation is absolutely not the reason they're good. It's not necessarily even done on a more capable model. It can even be done on itself and still bring improvement, or on a weaker model as well (see GLM and Gemini, which is definitely true because it repeats Deepmind's injections).

[parineum]

If distillation doesn't make them better, why do it?

If using Anthropic models for distillation doesn't make them better, why do it?

[orbital-decay]

It does, it's just not the reason. Most of the work is done before that point, and as I said z.ai used a weaker model. Besides, this is all strictly one-sided, as nobody knows how much Anthropic and OpenAI borrowed from Chinese labs' open research and weights (and they innovated a lot, to put it mildly, starting with first reasoning models worth talking about long before OAI did the same). Chinese labs are also severely restricted on hardware.

This entire story makes certain American AI shops look cartoonishly evil and Chinese ones relatively sane. Not only they want to grab without giving anything back, they also want to sabotage everyone else's AI research and do plenty of terrible things like media manipulation on the global scale and getting in bed with the government. This can't possibly end well, for the Americans in the first place.