[petilon]

Enormous amounts of money is being invested in the development of AI models. Investors expect returns on their investment or they will not continue investing. Open weights make it harder for investors to get their money back, so it harms the industry.

Once the weights are out, it makes no sense to ban them in the US while the rest of the world takes advantage of it. But that doesn't mean developers of frontier models shouldn't take steps to prevent their weights from being stolen.

[danny_codes]

Ridiculous. If investors wish to set their money on fire investing in over-valued companies, they are welcome to do so. Protectionism to preserve ROI is a dumb policy. Just look at the US car industry. We're building dinosaurs. On this trajectory we'll have 0% market share abroad in 10 years. Chinese EVs will probably get market share at a 100% tariff because US automakers fell so far behind.

Same situation in protectionism in AI. The rest of the world will simply lap us.

Another shill account I gues

[petilon]

FYI, developers of frontier models taking steps to prevent their weights from being stolen is not "protectionism". If you guard your wallet from being stolen is that protectionism?

[danny_codes]

Who cares? Distillation is in the same class of theft as Anthropic stealing their data in the first place. By all means, Anthropic can try to prevent distillation. It'll be like holding back a storm surge with a single bag of sand. At the very least it'll be entertaining.

Plus it's not like Anthropic has any technical advantage over the Chinese LLM providers, they just have access to more compute. Once sufficient compute becomes available it's nothing special, the math for LLMs is extremely basic, any reasonably trained CS masters/PhD student can ramp up to SOTA in a year or two.

[petilon]

Sorry, disagree. "any reasonably trained CS masters/PhD student"? That's funny. Musk and Zuckerberg have failed to build frontier models, and they have plenty of reasonably trained CS masters/PhD students.

Anthropic's advantage comes from being an American company. I'll leave you with this quote:

China can draw on a talent pool of 1.3 billion people, but the United States can draw on a talent pool of 7 billion and recombine them in a diverse culture that enhances creativity in a way that ethnic Han nationalism cannot. --Lee Kuan Yew

[jumpkick]

How are weights stolen from the frontier model developers? What is that actual mechanism?

[eigenspace]

The weights themselves aren't stolen. The claim is that Chinese companies are using VPNs and proxies to buy massive amounts of Claude Pro and Codex subscription accounts, and then selling usage on those subscriptions as cheap white-label LLM API usage.

While selling that LLM API usage, they then capture all the prompts, outputs, and intermediate thinking the LLM does, and then sell those logs to the companies making open-weight models. The open-weight model developers then train on those logs to 'distill' a model.

[ForHackernews]

...good for them?

We used to call that competition.

Imagine making this argument with a straight face in any other industry:

"The claim is that Japanese car companies are buying Ford vehicles, and then leasing them to American consumers at cut-rate prices. In return for the cheap cars, the customers are letting the Japanese observe their driving behavior, studying how they use their F-150 and then the Japanese car companies are applying that data to design new vehicles that will directly replace Ford!"

[eigenspace]

I wasn't arguing against the practice, I was clarifying that the weights of these models are not being stolen, and explaining what these companies do to create their models.

[petilon]

> observe their driving behavior

That's not what they are observing, it is the behavior of the engines.

[ForHackernews]

:sigh:

fine, s/observing the way the Ford cars work

[jumpkick]

The parent I replied to said frontier weights were “being stolen” which is not the literal case. I think precision is important here.

[eigenspace]

I agree. That's why I said that the weights were not being stolen, and then explained what is actually being done.

[singingtoday]

Fantastic, thank you for the explanation!