[Tadpole9181]

Can you please stop being coy and intentionally obtuse? Just have a discussion in good faith, I'm so beyond sick of this kind of rhetoric.

Yes, AI training uses human data and a lot of it was not compensated. But that has absolutely nothing to do with the thing this thread is about.

Distilling models costs less money than a really procuring quality data and training a model yourself. If you disagree, debate that.

[lelanthran]

> But that has absolutely nothing to do with the thing this thread is about.

That's how this thread started:

>>>> That's the difference between innovating and copying/distilling someone else's innovation.

How is distillation by one party okay but distillation by another not okay, even though in the second case the other party is paying the asking fees?

[rmunn]

And, as I pointed out elsewhere, the difference I was pointing out was an economic difference. You keep on ignoring that fact and thinking I'm talking about ethics, but I'm not. I'm saying the difference is between spending trillions on training from raw data vs. spending billions on distilling that trained-from-raw-data model.

[lelanthran]

> And, as I pointed out elsewhere, the difference I was pointing out was an economic difference. You keep on ignoring that fact and thinking I'm talking about ethics, but I'm not.

I am talking about money now;

> . I'm saying the difference is between spending trillions on training from raw data vs. spending billions on distilling that trained-from-raw-data model.

Firstly, they didn't spend trillions on training.

I am pointing out that literal trillions were spent to assemble the data that the AI corps then spent dozens of billions training from.

You ignored that fact completely.

[rmunn]

> Firstly, they didn't spend trillions on training.

Then please argue with the guy who said they did, the guy who goes by richardw and whose comment I was responding to in the first place. He's the one who said "If the US companies need trillions to barely beat Chinese companies spending billions, despite a multi year head start..." and, to the best of my knowledge, you haven't replied to him at all. Instead, you have consistently replied to me, arguing points I'm not discussing, not TRYING to discuss, and arguing with me about points that somebody else made. As I told you in a different comment, PLEASE read more carefully in the future.

I am sick and tired of talking about this by now. I no longer care if you have a good point or not, you have exhausted my supply of good will. Good bye. Feel free to have the last word, you're the only one who cares by now.

[Tadpole9181]

First of all, this is actually how this part of the conversation started:

> If the US companies need trillions to barely beat Chinese companies spending billions, despite a multi year head start...

Because they're talking about the cost difference of distilled model development and ground-up trained model development.

And second, the answer is that OpenAI and Anthropic had to do all the research into how to train models. Then they had to acquire all the data, curate and filter it. Then they had to design all the ways to iterate on training and antagonize it to be better - because there's not actually an enormous corpus of aligned, human stream-of-thought data. Then over half a decade they've been refining these methods.

There's no way you don't understand that if it was not easier and cheaper to distill a model, the institutions in question would be training their own models from scratch.