[m-i-l]

Yes, a community effort against the botnets would be great.

The "good bots" which identify themselves and follow the rules are easy enough to block, so not a problem. It is the "bad bots" which pretend to be real users and hide behind residential proxies, and so are almost impossible to block at the moment, which are the problem.

Given that there are big companies openly (i.e. on the clearweb, not even darkweb) selling access to these residential proxy botnets of compromised smart TVs[0] and mobile phones and other devices, can we not simply get a database of the these IPs and block access from them? I would venture that almost every single one of those residential users are unaware that they have devices in their homes which have been compromised and are being abused in this way, so if they were to start seeing messages from more and more sites along the lines of "Access to this site has been blocked because unusual traffic has been detected from your computer network. Please check all devices on your network and remove any malware which may be routing this traffic." then maybe we could start addressing the problem at the source.

[0] https://news.ycombinator.com/item?id=49000864

[TeMPOraL]

> The "good bots" which identify themselves and follow the rules are easy enough to block, so not a problem.

And then they get blocked, which is a problem. In particular, the modern agentic AI tools interacting with web services to fulfill user queries - they are acting as user agents, and they should not be discriminated against.

So I'd say the first pattern that needs to be broadly adopted is non-discrimination of user agents.

But of course we've tried that in the past, the whole problem is that non-browser user agents == end-user automation, which is anathema to pretty much every on-line business out there, as money made online is primarily conditioned on users wasting their own lives on interacting with services directly.