[latch]

We were handling over 10K req / sec through that layer of our system and load tested the entire system to over 100K, testing various cache hit ratios (our 95th percentile uncached response time was ~5ms for our core service). At that scale, a sudden 400x spike isn't something you worry about -- our various network providers would null route us way before we ever got there.

When we moved away from Varnish towards our own integrate cache, our hit ratio went from 50% to 80% on average, and some individual routes hit over 95% (we proactively purged+refetched in the background through a queue).

So maybe you're right, but for the opposite reason that you state. Our scale and performance requirements were so great, a custom solution made more sense.

[ruben_varnish]

I think that the point with Varnish and VCL is that you can adapt it, tailor it and even extend it (the language with additional functionality from external libraries i.e. cURL or memcached) to the point that it fits your system just the way you need it.

Obviously, depending on what you were trying to do and to which extend Varnish was extendable then (VMODs were added in 3.0) it might not have been the best option there is.

Anyway, thank you for your reply. It was a very interesting insight you came with there.