[likeabatterycar]

> For one thing it's trivial for malicious actors to change their user-agent.

Absolutely true. But the programmers of these bots are lazy and often don't. So if Cloudflare has access to other data that can positively identify bots, and there is a high correlation with a particular user agent, well then it's a good first-pass indication despite collateral damage from false positives.

[plaguuuuuu]

The programmers of these bots are not lazy - this space is a thriving industry with a bunch of commercial bots, the abiluty of whcih to evade cloudflare/etc is the literal metric that determines their commercial viability

[likeabatterycar]

My data says otherwise and you have provided nothing to back up your claim other than saying we have an industry full of dirty money paying programmers to write unethical code. I'm sure it inspires them to do their best work.

Half these imbeciles don't even change the user-agent from the scraper they downloaded off GitHub.

I employ lots of filtering so it's possible the data is skewed towards those that sneak through the sieve - but they've already been caught, so it's meaningless.

[ok_dad]

I would hope Cloudflare would be way, way beyond a “first pass” at this stuff. That’s logic you use for a ten person startup, not the company who’s managed to capture the fucking internet under their network.

[sangnoir]

> So if Cloudflare has access to other data that can positively identify bots

They do not - not definitively [1]. This cat-and-mouse game is stochastic at higher levels, with bots doing their best to blend in with regular traffic, and the defense trying to pick up signals barely above the noise floor. There are diminishing returns to battling bots that are indistinguishable from regular users.

1. A few weeks ago, the HN frontpage had a browser-based project that claimed to be undetectable

[fbrchps]

> a browser-based project that claimed to be undetectable

For now

[sangnoir]

That's just part of the game. Sometimes you're ahead, sometimes you're behind, but there's never a decisive winner.