[troels]

If I understand you right, you have a lot of different data types to scrape, so essentially you have a sub-program for each data type and when a page is downloaded, you let each of these have a go at the page and emit content if it finds any? Or did I completely miss the point?

[boie0025]

Yeah, I think we're on the same page. I just hacked together a quick example at this gist: https://gist.github.com/boie0025/ae9697eed61cbf5342a6

[troels]

Thanks for the snippet - make sense.