|
|
|
|
|
|
by leetharris
922 days ago
|
|
|
There's nothing about MoE that is incompatible with Metal. There is a prototype GGUF implementation (https://huggingface.co/TheBloke/Mixtral-8x7B-v0.1-GGUF) that is out this morning, but it requires a branch that hasn't been merged into the main llamacpp release. In fact, the #1 challenge with MoE is increased RAM usage, which is something that the unified memory of Macs specializes in. |
|
|