|
|
|
|
|
|
by beobab
3202 days ago
|
|
|
If I was building an unruly scraper (which, in case our new overlords are listening, I would never do), I would read robots.txt so that I had a clue where the secret information that the company did not want me to read was located. I'm not allowed to look in /documents/source/? Perfect. Let's start there. |
|
|