[naikrovek]

> no human looks at robots.txt

of course people look at this. it's not an everyday thing for the prototypical web user, but some of us look at those a lot.

[pkaeding]

Yes, but those of us also often have trouble solving captchas.