[xtracto]

Shit... I was just looking at the DeepSeekMoE paper, almost exactly from 1 year ago:

https://arxiv.org/html/2401.06066v1

The red star in the first chart is so funny.