[IX-103]

Doesn't that mean these 33k tokens can be cached, since they don't depend on the input? The model can just start generation on the 33k+1th token.

[mixermachine]

right, but the cache retention time is very short for Anthropics LLMs. 5 minutes or 1 hour (with additional costs). So you have to prompt basically non stop to not get a cache eviction.

Anthropic even changed this silently: https://www.reddit.com/r/ClaudeAI/comments/1sk3m12/followup_...

[NiloCK]

A const prompt across all of Anthropic's subscribers could draw from a global cache rather than per-user?

Although saying that out loud makes me question it - each per-user chat and growing cache would need eventually to own its own ~contiguous memory block.

[mixermachine]

If there are no changes to the system message, yes this is possible and also likely done by Anthropic. When there are additional local MCPs, reusability will be lower.

I think that Anthropic will bill you in any case :D

[0-_-0]

Yes, but all subsequent tokens do depend on them, so they are still being charged for