Hacker News new | ask | show | jobs
by nython 2401 days ago
> As someone with lots of anti-anti-botting knowledge

Any recommendations for books/other information sources? Currently doing some backend work for a company that mainly does scraping and a lot of this seems to be based on the tribal knowledge of the resident old wise one.

1 comments

I sorta disagree on this being tribal knowledge - a lot of this stuff is out there if you're willing to dig a bit. Tons of it comes down to network reputation and having a legitimate-looking bot. If you're scraping at scale it's an infrastructure problem just as much as it is a fingerprinting one.

Stuff like https://news.ycombinator.com/item?id=20479015 is a goldmine for me and it usually is a fun weekend working around said methods in a lab-like environment.

> resident old wise one

He's just invested the time in picking this skillset up - it's definitely not just something you're an expert on after a few small projects. It takes years of having things break over, and over, and over [...]

There's a small Discord server that I setup for people who do a lot of RPA/web scraping if you're interested in joining a "tribe". My contact info is accessible through my profile if you're interested =)

re: tribal knowledge - didn't mean the field as a whole, just that bus factor is pretty low at the place I'm currently at.