[hiccuphippo]

Do AI crawlers actually follow robots.txt rules? They didn't care about rules when they trained them on copyrighted works so why would they follow the rules here?