Hacker News new | ask | show | jobs
by rzz3 311 days ago
Well Cloudflare doesn’t even block Google’s AI crawlers because they don’t differentiate themselves from their search crawlers. Cloudflare gives Google an unfair competitive advantage.
1 comments

Google claims their AI crawlers have user agents distinct from the search crawlers: https://developers.google.com/search/docs/crawling-indexing/...