[lemmsjid]

I’d say a useful way of thinking about caching is through the lens of the CAP theorem. You are facing a situation where compute requirements exceed the bounds of a single process. There are a variety of things you can do here, all with consequences to the Consistency aspect of your data. Two strategies with consequences are caching and horizontal scaling. So look to vertical scaling or efficiencies in data modeling first.

I like your comment btw. I’d add Observability to CAP to incorporate what you’re saying.