Hacker News new | ask | show | jobs
by olegus8 4033 days ago
Take a look at Weboob (http://weboob.org). It's an attempt to create a well-defined uniform API for websites lacking one. I wrote a few modules for it, and they're scraping 4 banks and 4 shops on a daily basis for last few months.

As I see it, upside of this approach is that you can scrape pretty much everything you can open in your browser. Downside is that you have to update your scraper every time website changes. From my personal experience this happens at least once a month.