[fiatjaf]

This is awesome. I've been dreaming about this for weeks.

I don't know if it is possible, but could this run as a Chrome Extension, in a background script, loading various pages, executing code on then and keep going, storing the data at the extension's localStorage?

It could also store the code of the scrapers, for reusing.

[fiatjaf]

Well, I see you already have almost all I suggested. Now I would want something to make the ajaxSpider render the pages using the browser engine, instead of just getting pure HTML.

[Yomguithereal]

This is an interesting point. I created an issue on the github repository concerning this matter. Maybe you'd like to comment on it about your use case so we can improve the tool?