Hacker News new | ask | show | jobs
by yalogin 4582 days ago
So what does it mean by facebook whitelisting a scraping service? Do they actively block scrapers?
1 comments

I could be wrong but I believe that the the default is that spiders are blocked and only the "User-Agents" listed are allowed to scrape (but not the disallow pages).
You are correct.