[nikon]

Agreed. I used BeautifulSoup in combination with Celery.

To me scraping is such a specific thing it's best to write your own 'framework'.