|
|
|
|
|
|
by staticman2
794 days ago
|
|
|
"For example 8x7B Mixtral only executes 13B parameters per token, but it behaves comparable to 34B and 70B models" Are you sure about that? I'm pretty sure Miqu (the leaked Mistral 70b model) is generally thought to be smarter than Mixtral 8x7b. |
|
|