Ask HN: How to wget pages with javascript?

[baconface]

I need to wget a url, allow the javascript to execute, and capture the result in a file. I've tried wget, curl, httrack,plugins like lget, and links(2) and none of them execute JS.

I want something like $: firefox <url> -s <file>

Does anyone know how to accomplish this behavior without using JS?

[dstein]

You can write a NodeJS script using the JSDom module to do this.

[baconface]

Thanks for the suggestion. Any idea how to accomplish this task without using JS?

I'm surprised chromium, konqueror and firefox don't offer this functionality for offline browsing.

[dstein]

Selenium (http://seleniumhq.org/) might work but it's not a simple command line script.

[bartonfink]

Selenium will do this, although it's not pretty. I spent about 4 months on a contract doing exactly that, and it sucked.

[a3camero]

iMacro? Rhino?

[profitbaron]

Personally, I'd use http://www.phantomjs.org/ and create a script which loads the page like a browser. Once you've done that you can get it to take screenshot or use JS to pull data off the page.

[baconface]

Thanks. I really want to get this done with the command line and it looks like Phantom JS will do the trick.

[ddemchuk]

perfect use case for celerity http://celerity.rubyforge.org/