[vancouvermatt]

Disclosure: I build a competing gateway.

We ended up doing roughly the session-lock you're describing — pin a conversation to whichever provider is already warm, reconsider only on a cold prefix — and what surprised me is that eviction was rarely the thing that broke it.

Any request that changed the prefix at all did: a tool definition added mid-session, a system prompt carrying a timestamp, retrieval that reorders context. Those all still look like a warm session to the router and are a full re-price in practice.

So the lock ended up being less "stay until eviction" and more "stay until something upstream invalidates the prefix" — and most of our savings came from stopping our own code from invalidating it, not from the routing decision.