|
|
|
|
|
by curio
5839 days ago
|
|
but their robots.txt says that you're welcome to scrape it: User-agent: *
Disallow: /ac.php
Disallow: /ae.php
Disallow: /album.php
Disallow: /ap.php
Disallow: /feeds/
Disallow: /o.php
Disallow: /p.php
Disallow: /photo_comments.php
Disallow: /photo_search.php
Disallow: /photos.php
|
|