Y
Hacker News
new
|
ask
|
show
|
jobs
by
fcarraldo
27 days ago
Does a mid-session provider switch result in loading the entire context into the new model, inflating session cost?
I don't think I understand the token/cost implications of this feature
2 comments
gunalx
27 days ago
Its nice if you used local, but needed å beefier modell, or more context Window. It will eat input tokens, but you do that all the time unless you have input caching.
link
resonious
27 days ago
Yes you pay a big burst right after switching. After that, everything is cached and it's smooth sailing.
link