Every site is different. Varnish is great for a particular fairly common use case, but it's not an all purpose "make site go faster" button. For example, putting varnish in front a CMS is usually a fantastic idea, or anything that has a high ratio of reads vs. writes and serves the same pages to multiple users. That could be anything from a content blog or site, an online store, etc. However, for other types of sites it doesn't make as much sense. A site like facebook or twitter would gain almost no advantage from it, since the overwhelmingly most common use case is for every single user to receive different pages on every single visit. Similarly, it doesn't make sense for search engines, or for web mail apps, etc.
Also, most really large sites have probably already developed some other method of caching if it suits their site needs, so it wouldn't make sense for them to switch over to varnish all of a sudden.
Facebook uses Varnish, so does Twitter. They use it where it makes sense, where reads are high and content is less dynamic. To say they'd gain almost no advantage of it is oversimplification as they have various requirements and some of those do indeed benefit from caching.
>Sites that use Varnish normally return the X-Varnish HTTP response header when you access the site. We used this as the indicator if a site is using Varnish or not, and scanned the response headers for the top 10,000 websites in the world according to Alexa.
Terrible methodology. I suspect it is vastly more popular than that methodology would lead you to believe.
Because X sucked. I hated X Windows, nothing worked right at all back then. Actually, the last time I used it, it still sucked. And how has X caught on?
I am on a dual screen X terminal, logged on a remote linux server (virtual quad core) and using rdesktop to access my desktop PC because I am not in my office.
In front of me, all the worktations (38) are running linux.
X windows is definitely the tool I use the most.
Very popular, to whom? Linux has what, 1% of the desktop market share. Linux is very popular on android, but I was pretty sure that Android didn't use X. That's like saying you were popular in High School because you had two friends. And apparently, from an article on HN, Ubuntu isn't going to be using it any further.
Varnish isn't life-changing software. One thing Poul-Henning Kamp is very good at is selling Varnish, but once you use it for a bit you'll understand why it's a pretty advanced tool that many people avoid.
I've deployed Varnish once and I was annoyed by how it is configured. PHK will, of course, spin that as a positive, but my preference is not to write C when configuring my infrastructure support.
Beyond that, caching is a complicated topic with specific nuances for every deployment. As an example, I work on a high-traffic social site and we have absolutely no need for Varnish or any software like it. At its essence, Varnish is fixing a problem that many sites (a) do not have (yet), or (b) fix in other ways. We don't cache our dynamic views at all, and our system keeps up just fine. When it doesn't, we fix the system. Caching is on the radar as an improvement but we have determined that in terms of reward it does not make sense for our environment (invalidation is too frequent).
If you find yourself needing Varnish, ask yourself why. The answer to that question might lead you down some things to fix before investing in a big cache tier. There's a reason Facebook uses Varnish and you're wondering why others don't.
Also, most really large sites have probably already developed some other method of caching if it suits their site needs, so it wouldn't make sense for them to switch over to varnish all of a sudden.