Hacker News new | ask | show | jobs
by M4v3R 912 days ago
Which honestly is the easiest option of them all if you own an Apple Silicon based Mac. You just download the ollama and then run `ollama run mixtral` (or choose a quantization from their models page if you don't have enough ram to run the defalt q4 model) and that's it.
1 comments

I tried an hour ago and had a can't load model error. Everything up to date. Is there any special step?
Tried `ollama pull mixtral` just now and it seems to be working, albeit pretty slowly.
How much RAM do you have? Mixtral is a beast and the non quantized model wants 40GB+ of memory.
Ah, that might be it! I have only 32
The q2 should fit.