[bitboop]

Would it be feasible to do some "intelligent" TTL updates? By which I mean, don't let the cached data expire automatically. Instead, query a service that informs it of any outage on the backend. If an outage exists, the TTL is renewed. Otherwise, the TTL is expired.

For the `F5` spammers, they would be served with the stale data, until the backend recovers from the outage.

I'm sure I've missed some obvious reason why this isn't a good idea. I'd love to know why.

[thehappypm]

I think the problem is the service that feeds the cache durations is something that can itself have an outage, which itself could trigger a cache stampede. Usually to solve these kinds of problems you want less new blocks in the block diagram, not more.

[cestith]

If you don't mind serving stale content, you can do exactly the opposite. Instead of a relatively short TTL that gets extended for failure one can set an infinite TTL and actively eject data that's been updated. In either case, having a separate service handle checking the query and timing out the cache content that is neither part of the frontend nor backend is another thing that can break but can be a fairly simple, isolated thing that doesn't break consistently along with either the frontend or backend.

[thehappypm]

This seems reasonable for a read-heavy system (like Reddit). In a failure, the content on the site would just get staler and staler, but there wouldn't be a heavy outage.

[jtsiskin]

This is like the “fail open vs fail closed” question. It depends on what is worse. For legal reasons, they might need the ability to say “yes, we can guarantee we will be no longer serving this content within 1 hour”.

[dna_polymerase]

I can already smell the startup that slaps a neural network in front of redis now. I hate that smell.