|
|
|
|
|
|
by marcyb5st
16 days ago
|
|
|
I believe part of the LLMOps (I don't like the term, but it is what it is) should be building a failover plan with proper testing that check tools trajectories and such. If you have these then you can sort the good enough models from cheaper to more expensive and have the failover you mentioned. I saw people bulding a mapping of model->{{prompts}, {tools descriptions}, ...}, but that, to me, it feels extreme. I believe it is the model that needs to adapt to your prompts after a certain point. Models that fail to do so won't get our api requests as they will be out of the failoever roster. |
|
|