[flowinghorse]

The number of disallow we found in the robots.txt files actually surprises us.

Companies like OpenAI also have to do a lot of things to ensure the compliance to the regulation.

[LinuxBender]

to ensure the compliance to the regulation

Did legislation pass requiring people and their bots to obey robots.txt? If so I totally missed it. That would be big news if so.