Hacker News new | ask | show | jobs
by riku_iki 1378 days ago
> Can anyone but google scrape the web? No. Tons of websites won't let you unless you are googlebot.

I though cloudfare has some way to onboard other bots too.