Hacker News new | ask | show | jobs
by varispeed 58 days ago
I find this 1M context bollocks. It's basically crap past 100k.
1 comments

I like not running into the mandatory compaction but I do try to actively keep it under too. From an Anthropic standpoint with the new(ish) 5min cache timeout, it's a great way to get people to burn tokens on reinitializing the cache without having them occupy TPU time.. Esp. the larger the context gets.