[jiehong]

So, using the smallest model for the task would help, as expected.

A very small model could run on device to automatically switch and choose the right model based on the request. It would help navigate the difficult naming of each model of each vendor for sure.

[potatolicious]

> ” A very small model could run on device to automatically switch and choose the right model based on the request.”

This is harder than it looks. A “router” model often has to be quite large to maintain routing accuracy, especially if you’re trying to understand regular user requests.

Small on-device models gating more powerful models most likely just leads to mis-routes.