| I can see how this makes sense as a default behavior for cost conscious users. I would prefer to have the option for my company to pay more to rehydrate the cache than to have there be a model performance difference when having idled for an hour. "We tried a few different approaches to improve this UX: 1. Educating users on X/social 2. Adding an in-product tip to recommend running /clear when re-visiting old conversations (we shipped a few iterations of this) 3. Eliding parts of the context after idle: old tool results, old messages, thinking. Of these, thinking performed the best, and when we shipped it, that's when we unintentionally introduced the bug in the blog post." I see how these interventions help users reduce their token burn rate, but they don't address the need for an enterprise user to maintain quality. A common workflow for me is kick off a prompt, commute home, eat dinner, follow up on prompt. Frequently 80K tokens or less in the context, frequently > 3 hours. Or when running multiple sessions it's easy to let a session idle for a few hours while I focus on one. Or many meetings might mean idle time for an hour. Also, for enterprise users, I don't think education on X is a great place. There are people upskilling on this that never intentionally go on X. First thing that comes to mind would be a weekly tip feed of footguns and underutilized functionality published to an anthropic website. "The Old New Thing" "Guru of the Week" "Abseil tips of the week" all have that format. |