[k9294]

What about request caching? If you swap to a cheaper model mid execution it might cost more that to make multiple requests to the already cached provider?

[adchurch]

Yep 100%, mentioned this in another thread (https://news.ycombinator.com/item?id=48689448) but tl;dr we build the router to be cache aware