Y
Hacker News
new
|
ask
|
show
|
jobs
by
TradingPlaces
676 days ago
I did not have great luck with 405b on an M1 Max Mac Studio with 64 GB of RAM via olama. I think memory was this issue. 70b runs with acceptable speed
1 comments
Lerc
676 days ago
64GB of ram is 550b
bits
of ram. That doesn't give you a lot of options on how many bits you want to keep per model parameter.
link