Hacker News new | ask | show | jobs
by TradingPlaces 676 days ago
I did not have great luck with 405b on an M1 Max Mac Studio with 64 GB of RAM via olama. I think memory was this issue. 70b runs with acceptable speed
1 comments

64GB of ram is 550b bits of ram. That doesn't give you a lot of options on how many bits you want to keep per model parameter.