[bbor]

Well, that's the meta-rub: if they're abusing, block abuse. Rate limits are far simpler, anyway!

In the interest of bringing the AI bickering to HN: I think one could accurately characterize "block bots just in case they choose to request too much data" as discrimination! Robots of course don't have any rights so it's not wrong, but it certainly might be unwise.

[inejge]

> Rate limits are far simpler, anyway!

Not when the bots are actively programmed to thwart them by using far-flung IP address carousels, request pacing, spoofed user agents and similar techniques. It's open war these days.

[parineum]

Request pacing sounds intentionally unabusive.

[j16sdiz]

It is not bringing down your server, but they are taking 80%+ of your bandwidth budget. Does this count as abuse?

[immibis]

Are you at a hoster with extortionately expensive bandwidth, such as AWS, GCP, or Azure?

[ithkuil]

Isn't that what a rate limiter would address?

[mkl]

Not when the traffic is coming from 10s of thousands of IP addresses, with very few requests from each one: https://drewdevault.com/2025/03/17/2025-03-17-Stop-externali...

[KronisLV]

That very much reads like the rant of someone who is sick and tired of the state of things.

I’m afraid that it doesn’t change anything in of itself and any sorts of solutions to only allow the users that you’re okay with are what’s direly needed all across the web.

Though reading about the people trying to mine crypto on a CI solution, it feels that sometimes it won’t just be LLM scrapers that you need to protect against but any number of malicious people.

At that point, you might as well run an invite only community.

[ithkuil]

ok, but my answer was about was how to react to request pacing.

If the abuser is using request pacing to make less request then that's making the abuser less abusive. If you're still complaining that request pacing is not pacing the requests down enough because the pacing is designed to just not bring your server down and instead make you consume money, then you can counteract that just by tuning the rate limiting even further down.

The 10s of thousands distinct IP address is another (and perfectly valid) issue, but it was not the point I answered to.

[rollcat]

It's called DDoS. DDoS is abusive.