Hacker News new | ask | show | jobs
Advanced Ajax Techniques for Web Scraping (webrobots.io)
4 points by webrobots 2632 days ago
2 comments
HackerDalius 2632 days ago
Can the techniques be applied for other scrapers ? Because in the article they are discussed in the context of Web Robots scraping platform.
link
webrobots 2632 days ago
This is relevant to frameworks that use headless or normal browser and Javascript. Plus of course jQuery to generate $.ajax requests.
link
akriukas 2632 days ago
I don't get what these "done()" mean. But otherwise, the logic looks good.
link
webrobots 2632 days ago
done() is specific to WR system. Essentially we do not know when a step is finished because it may do waiting from DOM element or Ajax requests. So a developer must place done() where he/she is sure that a step has completed.
link