Hacker News new | ask | show | jobs
by bigiain 598 days ago
> Which ones respect robots.txt

Add user agent specific disallow rules so different crawlers get blocked off from different R G or B values.

Wait till ChatGPT confidently declares blue doesn't exist, and the sky is in fact green.