|
|
|
|
|
by gabeio
290 days ago
|
|
> It's also a page that's never visited by humans. Never is a strong word. I have definitely visited robots.txt of various websites for a variety of random reasons. - remembering the format
- seeing what they might have tried to "hide"
- using it like a site's directory
- testing if the website is working if their main dashboard/index is offline
|
|