[RuiWang0811]

Does this mean you have to retrain routing rules every time a new model gets released? I imagine since the price/token (or rather the amount of work that can be done per token) does not monotonically increase with new models, that the routing logic has to change all the time

[rohaga]

Yeah this is a great observation--we have to collect some more data to understand new models as they come out. However, by having this composable architecture that separates "confidence prediction" and the actual "routing layer," this makes it a lot easier to add new models/restrict the models rather than re-training from scratch.

[Mzzzzz]

test