[trinovantes]

I wonder if fake bot detectors can distinguish between any Google IP like GCP instances (i.e. do they simply check the ASN) or crawler specific IPs

Or maybe google crawler also runs on GCP and it's indistinguishable from regular $5 compute users

[windowsworkstoo]

Yes, Google and most major search engines enable a RDNS lookup to validate they are really a googlebot

[trinovantes]

TIL. Found this for googlebot

https://developers.google.com/static/search/apis/ipranges/go...