Hacker News new | ask | show | jobs
by JohnFen 1031 days ago
But you and I aren't judges in a court. They go by different rules, such as the official intent and meaning of the robots.txt system itself.

I'm not saying a court wouldn't find intent signaled, I don't know, only that it's not clear-cut that it would.

1 comments

Aside from whether intent is signalled or not, I would imagine courts may want to identify whether it is reasonable for any particular intent to not be honored in any particular set of circumstances. As you say, robots.txt isn't mandatory to be honored by itself. Perhaps other things might make it so? I don't know.