We've been long vacillating on using Varnish to power our E-commerce store but from what we've read, Varnish is not a good solution in cases where cookies are part of most of web traffic. Is this true?
Varnish is extremely flexible. Its primary design goal is to be a programmable reverse proxy cache -- so it's almost certainly a better choice for a challenging session-dependent use case such as e-commerce than repurposed web servers (such as Apache, nginx, etc).
If every single dynamic page of your site requires knowledge of the session, and must hit the backend, then you're kind of stuck. But there are lots of ways to design around that.
For example... Edge Side Includes, supported by Varnish, let you do front-end caching of page fragments, stitched together by the proxy rather than the web application. This can give you a huge boost (to performance and scale), even if you have to hit the web application for some part of the page.
Keep in mind, once you start adding front-end caching to your application, you need to think of your cache as part of your app. :-)
You might want to take a look at the Turpentine module for Magento, which tightly integrates Varnish with Magento. It's a good example of retrofitting front-end caching, cookie generation, and ESI fragment caching into an existing e-commerce application.
Every site has something that can be cached. Images, js, headers, popular product lists, etc.
If a specific page item can be cached is really up to if the backend application takes cookie state into account when the page item is made.
The Varnish default of not caching anything when cookies are present is because we have no idea if the backend writes "Welcome back, krat0sprakhar!" when it sees the username in an incoming cookie. Sending that item to every user would be unfortunate.
Other than that, I'd recommend you evaluate your needs and not just pick a technology. If your page response times are low enough already, and your backends/appservers scale to as many concurrent buyers you think you need, you're good without Varnish/caching.
I'd add that edge side includes (https://www.varnish-cache.org/trac/wiki/ESIfeatures) can be amazingly useful. Even if you've offloaded all of your static content to a CDN, the ESI features of varnish alone can be a huge win.
This is false. Default configuration does not cache any pages with cookies. You can configure varnish to ignore cookies. But as others have stated, Varnish can not magically guess which cookies to ignore and which will modify your content. If your site has /admin, you can tell varnish to not cache that. The cleanest solution is to never ever set any cookies for url-s you want to cache. So if you set a login cookie you set it to /admin.
In addition to other answers - it depends what the cookies are for -- if your users only get a session cookie after logging in (before that they only have front-end cookies for things like google analytics / remembering which widget on the site is currently active / etc) then you can ignore front-end cookies, thus successfully caching for all not-logged-in users:
if(req.http.Cookie) {
# ignore front-end cookies (prefixed with "ui-")
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(ui-[a-z-]+)=[^;]*", "");
# ignore google cookies
set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js)=[^;]*", "");
# Remove a ";" prefix, if present.
set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
# Remove empty cookies.
if (req.http.Cookie ~ "^\s*$") {
unset req.http.Cookie;
}
}
In my read-mostly use-case, this allows ~90% of site traffic to be served by varnish, even though the site is totally dynamic (for not-logged-in users, all the state that we care about is in the GET parameters).
I just throw away the default config and write a vcl to do what I want.
You can cache while using cookies. If you use a cookie to change the page in any way add Cookie to you Vary headers. In Varnish I don't cache response with Vary ~ "Cookie". I also don't cache any response that sets a cookie.
Now if you have a backend that does modify the page using what is in a cookie but doesn't set the Vary header, then using Varnish can be quite a pain. This is hardly Varnish's fault that the backend is crappy.
This is false. I have personally configured Varnish to power blogs, marketplace websites, and a Magento powered E-commerce store.
If you've worked with Magento then you know it relies heavily on cookies and sessions. Even with all difficulties of modifying Magento I was able configure Magento and Varnish to work together perfectly within a week.
I can point you to the Magento site running with Varnish if you send me an email.
You can use Edge Side Includes to get around this. Essentially, you write a page to Varnish containing ESI tags, and it caches that page. The ESI tags, otoh, are parsed every time the page is accessed.
So it's useful if some amount of your content is always the same, with other parts being requested per request (or cached depending on cookies, or whatever else).
If your site is not completely dynamic there is no sane reason to hit the database or disk for every visitor. Serve it out of the cache. Improve your visitors' experience, and lower the load on your hardware to insignificant levels.
If every single dynamic page of your site requires knowledge of the session, and must hit the backend, then you're kind of stuck. But there are lots of ways to design around that.
For example... Edge Side Includes, supported by Varnish, let you do front-end caching of page fragments, stitched together by the proxy rather than the web application. This can give you a huge boost (to performance and scale), even if you have to hit the web application for some part of the page.
Keep in mind, once you start adding front-end caching to your application, you need to think of your cache as part of your app. :-)
You might want to take a look at the Turpentine module for Magento, which tightly integrates Varnish with Magento. It's a good example of retrofitting front-end caching, cookie generation, and ESI fragment caching into an existing e-commerce application.