Hacker News new | ask | show | jobs
by Kesty 3058 days ago
Google has checks in place to see if someone is serving things to GoogleBot differently that the rest of the users. So it almost definitely has bots that double checks pages without the user-agent.

If the "disguised" googlebot is the same as the actual one, chances are it is since it would want to be as close as possible to not flag false positives, and use the same seed for consistency then you might be able to use that to avoid detection on the fact that you are serving google something different than normal users.

Newspaper used/do that to be able to have their full article content indexed while serving a paywall to everyone else.