Hacker News new | ask | show | jobs
by raminf 701 days ago
FWIW, 405B not working with Ollama on a Mac M3-pro Max with 128GB RAM.

Times out.
1 comments
pbmonster 701 days ago
Did you get a 2 bit quant? You need to chain several Mac Studios via Exo to get enough memory for a useful quant to work.
link