Hacker News new | ask | show | jobs
by alt227 24 days ago
Yes I do, I also think they will continue identifying themselves in the user-agent header.

https://developers.google.com/crawling/docs/crawlers-fetcher...