Hacker News new | ask | show | jobs
by dvtkrlbs 38 days ago
Do you think the introduction of Anubis on a lot of open source websites was a coincidence. The AI companies' crawling bots don't play by the regular crawling rules and not a good citizen and they are causing a lot of issues. If your Claude session is using the same user agent of their data crawling bot (most of the time it will just check for claude in the user agent) yes you will be classified as bot as well.