[Anon1096]

Calling #2 more sustainable has no basis in reality, it's just a feeling. It's like saying that clothing before the loom or farming before the tractor were "more sustainable". No, it isn't, it just appeals to yeoman farmer instincts that somehow technology=bad when it's what powers (and sustains) our modern world of 8 billion people.

[tw04]

Given that #1 seems to be based almost entirely on stealing from #2, and never paying reparations, I’d say it’s pretty unsustainable.

It’s like saying robbing banks for a living isn’t sustainable and working at a bank is. That’s not exactly a stretch.

[fc417fc802]

#1 may well put #2 out of a living but that isn't the same as stealing and doesn't (at least in and of itself) make it unsustainable. The fact that models were trained on scraped content isn't a matter of technical necessity but rather the path of least resistance (lowest cost in this case). Synthetic data is increasingly used for reasons of quantity, quality, and various technical considerations.

[tw04]

All of the major players in AI currently, literally stole to build their models. There isn’t one out there that hasn’t. So yes, it is the same as stealing because they were LITERALLY, in the literal sense, stealing.

[fc417fc802]

Well, pirated. Piracy and stealing aren't the same thing.

Regardless, I acknowledged the general issue. However I pointed out that doing so was not a technical necessity. If you base your worldview or actions around X implying Y but then it turns out that actually Y was merely a matter of convenience you're probably going to arrive at a wrong conclusion.

There's also the issue where you're emphatically calling it stealing without providing a clear criteria. The legal system as a whole has yet to conclusively resolve the various piracy accusations. The legality of consuming publicly available content remains quite controversial.

[tw04]

It absolutely is a technical necessity. You could build a model from scratch today without doing the same thing. And every model attempting to train on AI generated output degrades into nonsense almost immediately.

There’s a reason Reddit is making millions of dollars letting these companies mine their human generated content. You think OpenAI or anyone else would pay for that if they could just cyclically train on AI generated content???

[fc417fc802]

> attempting to train on AI generated output

I said nothing about that. Good synthetic data does not (typically) involve ML algorithms. Although that might be changing.

I'll politely suggest that you go read the literature before engaging further.

Reddit, Twitter, and similar are valuable because the data covers current events. Their content makes up a reasonably comprehensive timeline of the world at large. You don't need that to train a barebones functional model but it's certainly useful in order to train a knowledgeable one. Regardless, if they're charging for access it clearly isn't piracy so it doesn't seem like your original objection would hold any water in that case.

[andyfilms1]

It's sustainable in the literal sense, I.E. a tailor can simply tailor forever without needing to constantly worry about keeping up with new tools or technologies, or needing to upgrade or change their methodology constantly.

The tech world is obsessed with moving fast and breaking things, and you can't just do the same thing forever and expect it to always work.

[Karrot_Kream]

Think about how much food we throw away in the developed and developing worlds. How often we buy new clothing when we could mend old clothing. How often we ask for more when we could do with less. How often we want to eat at a restaurant when we could make leftovers. How often we want something sweet when we could just eat something bland. How often we heat and cool our homes when we could wear more or less clothing.

It turns out that while these are all truisms, nobody wants to fix them. Developed countries are okay passing pigovian taxes, to a limited extent, to help fix these problems. Developing countries are even less interested in fixing these problems. It turns out that austerity is incredibly unpopular. Everyone wants to tell other people not to do the things they don't like but nobody wants to listen to what other people tell them not to do.

Just a reminder that Europe colonized Asia, Africa, and the Americas in the search for spices. Later on the interest changed to tea. Literally the only thing that Europe wanted was better tasting food and drink (initially at least.) By the time the potato had become widespread, they could have had enough calories to feed the continent, and yet the desire for flavor is what lead to untold misery for hundreds of years for millions of people.

We need to be realistic about what works and what doesn't. Austerity never wins.

[sonofhans]

“More sustainable” than burning hydrocarbons to produce chatbot tokens. Humanity could sustain itself on those resources much longer if we were more careful with them. The very definition of sustainability.

[bandrami]

The bigger issue is that the AI we currently have available has been produced by setting a giant pile of money on fire with no real plan for ever earning it back. And that is not sustainable because at some point the checks dry up. We have zero idea what the economics of unsubsidized inference are because we still don't actually know how much money the frontier labs are currently losing.

[phoronixrly]

It allows for our modern unsustainable world of 8 billion people you mean?