[dofm]

This is somewhat akin to the "one-expert-per-query" solution Apple are using in their small foundation models I think?