|
|
|
|
|
|
by p0u4a
1 day ago
|
|
|
> if software knows in advance which data chunk (expert) it'll need for the next token, it can load that in parallel with computing current token You could actually use the model's MTP head to make a ~decent prediction on what experts would be activate in future tokens and preload them |
|
|