[lukan]

"In MoE systems the routing decision is made per-token, not per prompt or task."

Have there been experiments with doing it per task? Like, "oh this is python project, use this model" "oh this is about writing fantasy, use this"?

[janalsncm]

It’s a good idea. The results probably depend a lot on how close your task is to the benchmarks though.

I think OpenAI already has (had?) a feature like this called “auto” mode for thinking.