Hacker News new | ask | show | jobs
by RaphiePS 4443 days ago
There are a bunch of comments about rolling your own scraper instead of relying upon a possibly unreliable SaaS app.

That makes me think -- would it be viable to run a service that, instead of running the scraping on their own servers, simply gave you a custom binary to run?

Assuming that you trusted the executable, you would never have to worry about the company failing. It'd just be a one-time fee, and yours to use in perpetuity. Presumably updates would be free.

2 comments

That's a really neat idea I'd pay for. Not sure about the sustainability of the model though.
if you use scrapy (which is an awesome python scraping framework) you can plug a different third party solutions such as: http://crawlera.com/

Not really server level hosting, but you get the benefits of their network.