[razodactyl]

Don't we pay for cache input though?

[broodbucket]

You can send and receive 1 byte and refresh the cache. Keeping the cache alive isn't free, but it's close.

[leemoore]

if you are refreshing a cache of 120k tokens, you have to input the same 120k tokens. you can't bounce the cache with 1 byte. A proper cache hit requires your entire context that is cached. if you change 1 byte in it, everything after that byte is a cache write and no longer a cache hit

[broodbucket]

Yes, you're paying the cache read, not the cache write, which is much more expensive. You can do the math for how many 5min refreshes you can be afk for until it starts costing instead of saving

[cortesoft]

The idea is this keeps it in cache so you don’t have to pay to re-input.