[amiga386]

This is absolutely fine, until some shitheel decides they're going to continually scrape your website, thousands of requests per second, and knock it offline constantly, so they don't have to cache even a single thing, or write better scrapers. They'll just suck up your bandwidth for it. After all, it's you that's paying, not them.

Search engines were absolutely fine. They were respectful of the sources they scraped. There's now a bunch of money-obsessed shitheels doing whatever they can, as blatantly and as lazily as they can, because they believe they'll become rich. Every one of these fuckers needs to be dead or in jail before the world becomes whole again.

If nobody takes action against these fuckers, everything you care about will just nope out of existence, and these fuckers will be all you have left.

[TeMPOraL]

Major AI companies aren't indiscriminately scrapping any more than search engines were (which themselves caused some problems in the past too, until that got sorted out). Your gripe is with SEO people and content marketers.

I'd be more sympathetic if it was really about this, though. However, the majority of the complaints seems to come from people personally offended by the possibility of their content, which they claimed was published for everyone to benefit, ending up as training data, and thus actually benefiting humanity, at scale far beyond the original publication ever would. Which is the Dog in the Manger attitude I point out.

[amiga386]

The problem is the rat race for the pot of gold at the end of the rainbow.

A similar thing happened when crypto was in ascendence. Web pages were getting crypto-miners injected into them. Celebrities were shilling NFTs. Everyone and their dog was on the make. The thought of riches broke the minds of millions.

The same is happening with AI. Whether it's the "major AI companies" or millions of self-interested also-rans with fewer moral scruples scraping, the problem still exists. The problem will continue to exist until you can't conceiveably make money by scraping like a bastard. If everybody identified themselves up front and respected robots.txt, there would not be a problem. But they don't, and they don't, and they pummel websites for no fucking reason, and they don't care, and they won't stop.

Websites get hammered by millions of unique IP addresses from residential ISPs which happen to belong to botnets, none of them identifying themselves as a bot user agent, all just pretending to be some slightly out-of-date version of Chrome.

And it's not like the "major AI companies" hands are clean. Who bought up all the RAM and compute power? Who's building datacenters in the third-world parts of the USA and powering 24/7 them with diesel generators, or sucking up most of the power generator, and of course using up all the drinking water to cool their machines?

Everything in humanity and nature is there for the AI bros to use and dispose of, provided they come out on top. This is the attitude that needs to be taken down.

[TeMPOraL]

You may be right about the scale of also-ran operations, even though I disagree with the core comparison. Unlike crypto coins, which require superlinearly growing energy waste just to sustain their basic guarantees, and were created to solve "problems" that aren't problems and don't need solving (hint: trust is a feature, not a bug), AI actually works. It delivers real value for cheap. The growth isn't artificial, it's a product that doesn't even need much marketing[0] - it exploded organically since ChatGPT, since it's obviously that immediately useful for approximately everyone in some aspects of their work or life.

One nit though:

> And it's not like the "major AI companies" hands are clean. Who bought up all the RAM and compute power? Who's building datacenters in the third-world parts of the USA and powering 24/7 them with diesel generators, or sucking up most of the power generator, and of course using up all the drinking water to cool their machines?

They paid for it, much like everyone else.

> Everything in humanity and nature is there for the AI bros to use and dispose of, provided they come out on top. This is the attitude that needs to be taken down.

Here you're arguing against the basic market economy. They aren't using and disposing of anything they couldn't buy for that purpose like literally everyone else. There's no theft or trickery going on here. There's a boom, because AI is that useful, but it's still all normal resource allocation.

--

[0] - Of course the competing players invest tons in marketing to gain an edge against the other players.

[amiga386]

> They paid for it, much like everyone else.

That's really not how it works. Did you miss the part where the RAM vendors, had they known their rivals were also selling all their RAM to OpenAI, would not have sold what they sold? OpenAI, who did know this, and what it would mean for every other industry and device on the planet that needs RAM, chose to go ahead with it. Because all they care about are themselves, fuck everyone else I got mine

Markets are imperfect because of information asymmetry, aka the Cheating Bastard Problem.

After announcing the deal and the truth coming out, they could also have said "hey, we see now this crimps all other industry but ours. We'll make this right, we'll reduce our RAM order if our vendors would prefer that." But they didn't, because fuck everyone else I got mine

As the AI companies are so flush with cash, they could afford to offer a Common Crawl, so everyone who wants web resources for their models or whatever can get very fresh cached copies from a common crawler, which obeys robots.txt and crawls in a fair way so as not to overwhelm the source webservers. But fuck everyone else I got mine, so here come a million dodgy scrapers to bring down everyone's site.

And in terms of natural resources, one does not simply "pay" and get whatever they want. Any government worthy of the name puts its own residents needs above the lucrative offers made to them by amoral outsiders who couldn't care less if the residents lived or died.

If you're a consumer of products and you know the product was made by taking away vital resources from people elsewhere, then you're complicit in that action by continuing to buy the product.

[hack1312]

Your rhetoric is harsh but you’re not wrong.