[janci]

Parsing the html or traversing DOM is the easy part. Doing request queues, ip rotation, data quality management, exponential backoff etc. on scale is much harder.

[ziflex]

PRs are welcome :) There is gonna be a separate project within the organization that would do all these things and even more. It's just beginning :)