Hacker News new | ask | show | jobs
by jamesog 585 days ago
That's not from robots.txt, but their Bot Management feature which blocks things calling themselves Googlebot that don't come from known Google IPs.
1 comments
speedgoose 585 days ago
Are GCP IPs considered Google IPs?
link
jgrahamc 585 days ago
No.
link
judge2020 585 days ago
For reference https://developers.google.com/search/docs/crawling-indexing/...
link
crop_rotation 585 days ago
No I am very sure they are not.
link