Hacker News new | ask | show | jobs
by samaybhavsar 3508 days ago
PHP and Python. Current count - 902873

I have also scrapped all the websites from Quantcast but haven't scanned it yet.

The website is build using PHP including the crawler which works for the new websites not in the list.

For scanning multiple websites, the crawler is in Python which uses multiprocessing.