[epolanski]

The fact that are people that genuinely believe you can train an LLM by using random QAs obtained from another LLM is astonishing. Let alone the fact that it makes absolutely zero financial sense.

At this point this is being repeated so often that completely uninformed users are taking this at face value.

[overfeed]

To be fair, Anthropic is participating in the misinformation by dishonestly characterizing what Alibaba is doing with the data as "Distillation" rather than the more probable adding a small fraction to the "fine-tuning" and/or benchmarking data sets.

I understand why - the distillation narrative casts Qwen as a poor copy of a superior model, and cultivates ground for political lobbying for bans. That doesn't make it less dishonest, but I suppose profits trump ethics.