[celdon25]

That doesn’t change anything regarding the actual point of the comment.

[astrange]

Your idea for a search competitor is to ignore robots.txt?

[VWWHFSfQ]

or an advertising competitor that ignores DNT!

oh wait

[celdon25]

Yes, clearly that's the best possible interpretation of what I said. /s

[chihuahua]

This seems perfectly logical to some people:

Google ignores robots.txt: "Google is evil! They're trespassing on my webserver!"

Google follows robots.txt: "Google search results suck! They're not indexing GitHub!"

[berkle4455]

It's a just a text file.