[cortesoft]

How do you calculate the effect of caching?

Sometimes, I take breaks in the middle of a session and end up losing the prompt cache which drives up the token usage a ton. Don't all the providers have different cache times and behavior? If one person takes 20 minutes between messages, some services will keep that cache while some won't. Is there a way to factor that in?

[schleck8]

OpenRouter shows the cache hit rate and calculates an effective pricing per provider if you scroll down on the model page. Note that some providers run fp4 versions of models. DeepInfra often does this

[ahmed_hassan7]

Author here — great question, and you've hit the exact reason the calculator uses an adjustable cache-hit ratio (default ~70%) instead of assuming 100%: real hit rates depend on session gaps, and TTLs differ a lot by provider.

Rough current state: OpenAI's automatic caching typically keeps prefixes warm for ~5–60 min depending on load; Anthropic's default cache TTL is 5 min (extendable to 1h for an extra write fee); Gemini's explicit caches let you set your own TTL with per-hour storage pricing. So your 20-minute coffee break is usually fine on OpenAI, a cold cache on default Anthropic, and whatever you paid for on Gemini.

Adding an "average gap between messages" input that derates the hit rate per provider is a really good idea — putting it on the list. Thanks!

[pigeons]

so many people write with your style now