[bitshiftfaced]

I wonder if Google sees MoE as a sort of local maxima, and so they tried a different path hoping it might outperform it.