[cjbarber]

This (agent detection) is now a kind of emerging space. Obviously it'll get much more important, too.

Other products in the space:

- Foil (https://usefoil.com/), I'm biased, a friend is building this

- Kasada https://www.kasada.io/

- DataDome (https://datadome.co/)

- Castle (https://castle.io/)

- Fingerprint (https://fingerprint.com/)

- HUMAN (http://humansecurity.com/)

- Google Cloud Fraud Defense, which is basically the updated reCaptcha (https://cloud.google.com/security/products/fraud-defense?hl=...)

- this, Cloudflare Precursor

It seems like some of the main reasons people care so far are:

- Preventing automated credential stuffing

- Preventing bots from creating a bunch of fake accounts (eg free trial abuse, which can also lead to high twilio SMS bills!)

- Reducing payment fraud

- Blocking LLM scraping

- Blocking automated scalpers (!) eg for tickers or sneakers

I'm curious to see which use cases end up dominating as the reason companies care about this. And I'm hopeful that my agents will still have good ways for me to browse and do things on the web on my behalf - eg detect agents and route them to an agent path, rather than blocking them.

(I'm interested in tools for detecting AI agents and seeing how this shifts as bot traffic goes way up.)

[lukewarm707]

happy to offer a counter of some great products for anti-bot defeat:

https://brightdata.com/

https://www.zenrows.com/

https://www.capsolver.com/

https://scrapfly.io/

hundreds of millions of residential ips, human browser fingerprints, custom browser binaries, auto solve of turnstyle, recaptcha v3, kasada, datadome, AWS WAF, etc if they come up.

[cjbarber]

Bright Data ranks #1 on Foil's leaderboard [1], but is still detected. ScrapFly is #4 and ZenRows #7. And I guess Capsolver isn't really a scraping thing but is more just for the captcha component.

I think it'd be good if there were more products that did a better job of making an actually undetectable agent, but doesn't seem like any exist yet.

[1]: https://usefoil.com/research/stealth-browser-leaderboard

[lukewarm707]

so, i found out why foil detects all the bypass products and the others fail badly:

it costs $0.05 per bot check

[lukewarm707]

i guess more people should use foil...big recaptcha, cloudflare challenge/turnstyle etc is 95+% bypassed, take your bets on how long this new cloudflare holds out

[harrylepotter]

Darwinium (darwinium.com) is another example. Approach here involves a combination of profiling and step-transition probabilities; idea is that a customer can ring-fence a particular area of a digital estate where they might want to challenge or block an agent - eg a payment, due to chargeback risks. Precursor at least for now seems more focused on site scraping multiple docs from the same site.