[greglindahl]

If crawling with JS costs 1,000X or 10,000X as much as crawling without, it's fair to say that even Google isn't going to crawl 100s of billions of pages executing JS.

As a former web-scale search engine CTO, my opinions are commonly surprising to folks who haven't built a web-scale crawler/search engine.

[tracker1]

My own experiments/experience shows that recrawls happen about 1/3 as often and tend to lag a few days behind for JS content vs inlined/delivered content. It's helped a little by dynamically delivering the sitemap data, but even that only speeds things up a little.

My guess is they're putting about 1/10th the effort into keeping things freshly indexed for JS, but may well be devoting 2x the resources vs directly received content.