[karaterobot]

> Also why are "AI" crawlers are worse than "normal" crawlers?

A search engine will index your content to bring people to it through search. An AI crawler will take your content to recapitulate it and sell it to others. Obviously it's more complicated than this, but this is how one might see it who wishes to use this file.

> Either way, this is an exercise in futility.

Not necessarily disqualifying. Laws against theft are also futile, in the sense that honest people don't need them and dishonest people don't follow them, and history since at least Hammurabi has been replete with examples of such laws not stopping theft. And yet. Seems worth the calories it costs to say "for the record, I do not give my consent for what you're doing".

[cabirum]

Search engines are not the beacons of holiness - they sell ads, they sell data on who searched what, they manipulate results.

Search engines and AI things are typically owned by the same company. AIs are fed with the data collected by a search engine. The only difference is whether AI gets the data in realtime or waits for the search engine to collect another data dump.

Fighting windmills as I see it.

[nunez]

Search engines manipulate results way less aggressively than LLMs do.