Hacker News new | ask | show | jobs
by dmitrysergeyev 1045 days ago
They could in theory combat it by comparing results with a second crawler that uses a different User Agent.
1 comments
mdaniel 1045 days ago
If they were going to the amount of energy to do a 2nd crawl using a different user agent, then why bother advertising the user agent at all and just feed it the Chrome one like every other home-grown spider does
link