Hacker News new | ask | show | jobs
by sephware 2818 days ago
"Much more effort"? Right now it implements a library and a language on top of that. Making it just be a library would cut the work in half.
1 comments

The idea is to create a high level abstraction that represents your web scraping logic. The project is still WIP. I will create a web server which will help you to store your queries, schedule them and set up output stream to other systems like Spark and Flink.