[zeras]

I think a fundamental mistake I see many developers make is they use caching trying to solve problems rather than improve efficiency.

It's the equivalent of adding more RAM to fix poor memory management or adding more CPUs/servers to compensate for resource heavy and slow requests and complex queries.

If your application requires caching to function effectively then you have a core issue that needs to be resolved, and if you don't address that issue then caching will become the problem eventually as your application grows more complex and active.

[chamomeal]

Idk I think caching is a crucial part of many well-designed systems. There’s a lot of very cache-able data out there. If invalidating events are well defined or the data is fine being stale (week/month level dashboards, for example), that’s a fantastic reason to use a cache. I’d much rather just stuff those values in a cache than figure out any other more complicated solution.

I also just think it’s a necessary evil of big systems. Sometimes you need derived data. You can even think about databases as a kind of cache: the “real” data is the stream of every event that ever updated data in the database! (Yes this stretching the meaning of cache lol)

However I agree that caching is often an easy bandaid for a bad architecture.

This talk on Apache Samza completely changed how I think about caching and derived data in general: https://youtu.be/fU9hR3kiOK0?si=t9IhfPtCsSyszscf

And this interview has some interesting insights on the problems that caching faces at super large scale systems (twitter specifically): https://softwareengineeringdaily.com/2023/01/12/caching-at-t...

[zeras]

Caching is definitely a useful and even a key component to producing efficent and high performance applications and services.

I think the mistake is not using caching, but rather using it too soon in the development process.

There are times when caching is a requirement because there is simply no way to provide efficient performance without it, but I think too many times developers jump straight to caching without thinking because it solves potential problems for them before they happen.

The real problem comes later though at scale when caching can no long compensate for the development inefficiencies.

Now the developers have to start rewriting core code which will take time to thoroughly complete and test and/or the engineers have to figure out a way to throw more resources at the problem.

[hinkley]

There are a lot of things necessary to be a successful human but doing them without doing the fundamentals just makes you a monkey in a suit.

Caching belongs at the end of a long development arc. And it will be the end whether you want it too or not. Adding caching is the beginning of the end of large architectural improvements, because caches jam up the analysis and testing infrastructure. Everything about improving or adding features to the code slows down, eventually to a crawl.

[hinkley]

> It's the equivalent of adding more RAM to fix poor memory management

No it’s ten times worse than that. Adding RAM doesn’t make the task of fixing the memory management problems intrinsically harder. It just makes the problem bigger when you do fix it.

Adding caching to your app makes all of the tools used for detecting and categorizing performance issues much harder to use. We already have too many developers and “engineers” who balk at learning more than the basics of using these tools. Caching is like stirring up sediment in a submarine cave. Now only the most disciplined can still function and often just barely.

When you don’t have caches, data has to flow along the call tree. So if you need a user’s data in three places, that data either flows to those three or you have to look it up three times, which can introduce concurrency issues if the user metadata changes in the middle of a request. But because it’s inefficient there is clear incentive to fix the data propagation issues. Fixing those issues will make testing easier because now the data is passed in instead of having to mock the lookup code.

Then you introduce caching. Now the incentive is mostly gone, since you will only improve cold start performance. And now there is a perverse incentive to never propagate the data again. You start moving backward. Soon there are eight places in the code that use that data, because looking it up was “free” and they are all detached from each other. And now you can’t even turn off the cache, and cache traffic doesn’t tell you what your costs are.

And because the lookup is “free” the user lookup code disappears from your perf data and flame graphs. Only a madman like me will still tackle such a mess, and even I have difficulty finding the motivation.

For these reasons I say with great confidence and no small authority: adding caching to your app is the last major performance improvement most teams will ever see. So if you reach for it prematurely, you’re stuck with what you’ve got. Now a more astute competitor can deliver a faster, cheaper, or both product that eats your lunch and your team will swear there is nothing they can do about it because the app is already as fast as they can make it, and here are the statistics that “prove” it.

Friends don’t let friends put caches on immature apps.

[lemmsjid]

I’d say a useful way of thinking about caching is through the lens of the CAP theorem. You are facing a situation where compute requirements exceed the bounds of a single process. There are a variety of things you can do here, all with consequences to the Consistency aspect of your data. Two strategies with consequences are caching and horizontal scaling. So look to vertical scaling or efficiencies in data modeling first.

I like your comment btw. I’d add Observability to CAP to incorporate what you’re saying.

[cortesoft]

> If your application requires caching to function effectively then you have a core issue that needs to be resolved, and if you don't address that issue then caching will become the problem eventually as your application grows more complex and active.

I don’t think this is always true. Sometimes your app simply has data that takes a lot of computation to generate but doesn’t need to be generated often. Any way you solve this is going to be able to be described as a ‘cache’ even if you are just storing calculations in your main database. That doesn’t mean your application has a fundamental design flaw, it could mean your use case has a fundamental cache requirement.

[mannyv]

If your database is slow because it's on spinning disks, then a cache will speed up access.

That's not a fundamental mistake, and there's very little you can do about that from an efficiency point of view.

It's easy to forget that there was a world without SSDs, high speed pipes, etc - but it actually did exist. And that wasn't so long ago either.

And of course sometimes putting data nearer to the user actually makes sense...like the Netflix movie boxes inside various POPs or CDNs. Bandwidth and latency are actual factors for many applications.

That said, most applications probably should investigate adding indexes to their databases (or noSQL databases) instead of adding a cache layer.

[jiggawatts]

Not to mention latency! Caching does nothing to fix the latency of “misses”, which means any app that uses a caching layer to paper over a bad design will forever have a terrible P99 (or even P90) latency.

“But, but, when I reload the page now it’s fast! I fixed it!”