|
|
|
|
|
|
by luckylion
2591 days ago
|
|
|
No need, you can block them by user agent, they are consistent - or just disallow them in robots.txt. If you need to make sure it's actually a google bot when a client shows up with the user agent, you can use reverse dns: https://support.google.com/webmasters/answer/80553 |
|
|