[omh]

When I visit the ICO site I also get a "civicCookieControl" cookie which you don't list. I guess that this is probably coming from javascript and your site isn't processing this?

[bartkappenburg]

It's running a headless browser with JS enabled so it should be there... not sure why it doesn't pick it up. Thanks for the mention :-)

[LunaSea]

If you are running PhantomJs, did you encounter any issues like memory leaks and socket errors ?

I'd be interested in the solutions you used to solve issues like that if it's not proprietary / private.

[ZirconCode]

Also interested, I've tried similar things with watir webdriver and a xvfb/headless mode but the only real 'solutions' I've come across are time-outs. Needless to say it could've been heaps better.

[LunaSea]

I didn't do much testing since PhantomJS 2.0 but I know that in the past, it was recommended to reload PhantomJS every +/- 50 pages to free the leaked memory.

[bartkappenburg]

We're running HTMLUnit [1]

[1] http://htmlunit.sourceforge.net/

[dspillett]

It could also be that the cookies handed out vary geographically, or if a page includes adverts in iframes via an ad provider network the cookies you get will depend which member of the network your request is redirected to this time.