[colinmhayes]

I built a search engine that was able to crawl Amazon and Yelp. The toughest sites were reddit and facebook.

[AznHisoka]

at scale? millions of pages a week? And now? I wrote a crawler that could crawl Amazon as early as a year ago too, but now it doesn't work.

[creato]

And google sucks at those too.