Hacker News new | ask | show | jobs
by ptr 3222 days ago
Why not just pull it out into a function, "loginWithForm(session, 'user', 'password')"? Why is it interesting on a class level that the scraper is using a form based authentication method? What if a page is switching between two authentication methods just for laughs?
1 comments

I think you've hit it on the head. Mostly you shouldn't be trying to codify what a scraper is, you should be collecting little utilities that are useful for building scrapers. The more lego-like the better.