|
|
|
|
|
|
by iot_devs
21 days ago
|
|
|
I am afraid that this post is missing the biggest point. Given a prompt (or task) how do I evaluate if it is a "simple" task that should be executed by a small model or if it is a complex one that may need a SOTA model? Are you guys using heuristics? Which one? |
|
|
When you use Pi with pi-role-model, Pi will include task and role metadata with its request; the role-model router runtime additionally holds benchmark and observability data, and a configured routing strategy. A composite of this is used to make the decision.
What you are pointing out is correct: making the actual decision and ensuring it is accurate can be difficult, which is why you need rich data as above, and also a model pool where each model is distinct, as I wrote in this post and in one of the comments below.