[nraford]

How did you get Mixtral to run an a 32gb M1?

I tried using Ollama on my machine (same specs as above) and it told me I needed 49gb RAM minimum.

[eurekin]

I'm using:

  ollama run dolphin-mixtral:8x7b-v2.5-q3_K_S

[mark_l_watson]

That runs on 32G? The original mixtral q3 wouldn’t run for me. Maybe the dolphin tuned version is smaller?

EDIT: I just checked, it runs great, thanks.