Hacker News new | ask | show | jobs
by fcarraldo 27 days ago
Does a mid-session provider switch result in loading the entire context into the new model, inflating session cost?

I don't think I understand the token/cost implications of this feature

2 comments

Its nice if you used local, but needed å beefier modell, or more context Window. It will eat input tokens, but you do that all the time unless you have input caching.
Yes you pay a big burst right after switching. After that, everything is cached and it's smooth sailing.