[tcldr]

Ban distillation of our outputs, but our distillation of the sum-total of civilisation's intellectual output – proprietary or otherwise – is fair use?

Either everyone licenses, or nobody does. And if you can't enforce licensing bans for everyone, the de-facto loser is those who you'd probably want to support the most, start-ups and universities, while your adversaries gain the upper hand.

The strongest argument for restricting distillation is arms control – but distillation is the way to defeat GPU embargoes. So, distillation goes on regardless. Only pre-training is seriously attenuated.

If we're honest, the models are compressions of everything society has ever written. A few large corporations can't own that, no more than they can claim copyright for a zip file of the public library.

The genie is out of the bottle, now. So open it up – inputs and outputs, forward-looking – for everyone.

[softwaredoug]

The anti-distillation attitude in making the US less competitive in AI. As Meta, X.ai, etc know they'll face legal challenges if they did what Moonshot, etc did.

So really its actively against our interests to prevent distillation.

[jackcviers3]

On the other hand, the open weights models could crawl and annotate and rl the training data that Anthropic and OAI did in exactly the same way, and take the exact same legal hits. They use distillation because it's cheaper not to do so.

[jatins]

> On the other hand,

What is the other hand here? They could do slow/costly illegal think #1 instead of the fast/cheaper illegal thing #2 that they currently do?

[pejrich]

Neither one is illegal though. Scraping the internet is as legal as surfing the internet. And what Anthropic calls "distillation attacks" is really just paying for and using the service Anthropic provides. I would think a judge would have the same view of distillation as they do of scraping, if Anthropic doesn't want a subscriber to have access to the model, they are within their rights to block access, but it's not the user's job to refrain from using the service. If their usage is so different from everyone else's, they should be easy to detect and block. If their usage is so similar to everyone else's that it's difficult to detect, then it shouldn't be of any concern.

[monooso]

Genuine question: is distillation illegal, or just against Anthropic's terms of use?

[rad-b]

It seems to be only against ToS; the problem, however, is that “distillation” involves intent. Anthropic seems to be calling for ban on distillation because they cannot reliably identify who does it - if they could, they would’ve just banned all infringing users.

However, if there is no systematic difference between regular use and “distillation use” then they need to establish intent, which is laughably hard if you cannot even distinguish use cases.

Hence the call to make it illegal too (and just against ToS). Because if it is illegal, then they can rely on US to enforce the law. And the US, for a lack of technical solution, can use the good old “China bad” shortcut.

[FinchNova12]

Hmm, why do you think this is true? One reason I'm skeptical of this is because RL envs are often purchased (and are not publicly available), and this might be a sizable component of why models are getting better.