Hacker News new | ask | show | jobs
by h0l0cube 1234 days ago
Certainly caching is important, especially for Word2Vec or other NLP which you'd want to happen in a separate stage after crawl, but as someone mentioned in a sibling comment, there are some factors that are calculated per-query, which can have a lot of cache misses for novel queries.
1 comments

If so, I'd highly suspect Google varies the compute/cache permitted for novel queries.

By this point, I can't imagine they haven't automatically balanced {revenueFromQuery} to {costOfQuery}.

No sense delivering hyperoptimized results if you lose money on generating them.

I’d suspect you’re right