Hacker News new | ask | show | jobs
by krakensden 5538 days ago
Scrapy is overkill for nearly everything. You'll probably have under a page of code using lxml and urllib.
1 comments
cdr 5538 days ago
I have under a page of code with Scrapy for simple projects, and more advanced features when I need them.

That's like saying "jQuery is overkill for just about everything, you should use plain javascript".

link
krakensden 5537 days ago
No, it's like saying "The full YUI suite is overkill for just about everything, you should just use the core or jQuery".

'scrapy startproject' creates a couple nested directories, with maybe seven files. Are you writing a scraper that you're going to run regularly? Does it need to be super robust and maintainable? Or are you writing something that you'll run once, maybe twice?

link
cdr 5529 days ago
I seem to be missing why you think using a framework is a bad thing. With say django or YUI there are performance and abstraction issues that can bite you, but I don't see those mattering for so lightweight a framework and tightly scoped a problem.
link