Hacker News new | ask | show | jobs
by irthomasthomas 424 days ago
I assume they use a conversation, so if you compress the prompt immediately you should only break cache once, and still hit cache on subsequent prompts?

So instead of Write Hit Hit Hit

It's Write Write Hit Hit Hit