[shakezula]

I’m not sure about “best” but I’ve been using Colly (written in Go) and it’s been pretty slick. Haven’t run in to anything it can’t do.

http://go-colly.org/

[nprateem]

I've used scrapy but I prefer colly. It's fast and simple.