Hacker News new | ask | show | jobs
by ivanca 4199 days ago
This is an interested idea, but what I would pay for is a GUI for scrapping, for example: clicking an element, seeing what classes it got, and be able to click each class to see if it matches the elements I need. And then generate the code for those actions I did in the GUI, maybe even be able to do both (modify the JS and still work with the GUI when required, yeah, is hard but possible)

The other major feature I need is paths of execution, for example if there are two possible pages after certain step (think if-else) I want those views visualized as interconnected nodes.

2 comments

If you are looking for a GUI for scraping, check out kimono labs. It's a GUI tool for scraping, and requires no code to set up and can find all similar elements from the one you click on. It supports pagination and other types of scrapes too.
Selenium could do that for you, and there's an IDE available as a Firefox plugin; branching might be tricky and you'd need to build some tooling around it in general, but it might give you a place to start.