[didibus]

> which is nonsense: the gate simply, directly, selects the experts. There's nothing probabilistic about it.

Isn't G a learned probability?

[wren6991]

I don't think so. Probability of what?

If you put something through a softmax the output is (trivially) a valid PMF. Does that matter? You're not sampling from it.