[oypass]

What is the benefits of on device routing? How do you decide if the task can be run on device?

[davidvgilmore]

For those that have capable enough hardware, it's effectively free to run subtasks on-device. (just the marginal cost of additional electricity).

With Google's most recent 12b param Gemma model, even Mac users with just 16gb of unified memory can offload some tasks on-device.