Hey. Thanks. Yeah I will add a ton of features over the next few days. JSONP should be an easy one. Feel free to add an issue in Github and I'll get it done for you.
Only really started hacking around on the idea the other day so early stages. Want to add filters so you can say "grab me only the text" or "grab me just the class names". Obviously another step would be to grab multiple elements in one request.
That's a good point. I pretty much wrote this in an evening or two so haven't had time to refine it much. But yeah error messages will definitely be improved. It's because of the way URLs are handled in the underlying web app. Will be an easy fix.
Cool idea. This could easily be extended to support something like a proxy pool; that way you can rate limit / rotate proxies for X domain globally at this server level. That way it's across all your projects, rather than having to do it on a per project basis.
Adding xPath support as well as CSS selectors would be a good addition.
Will definitely do something with caching and rate limiting when I get some time. These queries are quite expensive so definitely needs a bit of work in those areas.
Pretty interesting. Wrote a web scraping api you can paste in to your browser and download results last year but took it down to work on another project. You can take look at what a url could look like.
https://falkor-api.herokuapp.com/api/query?url=https://news....