Hacker News new | ask | show | jobs
by stavros 930 days ago
Yes, but I also care about "can I load this onto my home GPU?" where, if I need all experts for this to run, the answer is "no".
1 comments
MacsHeadroom 928 days ago
The answer is yes if you have a 24GB GPU. Just wait for 4bit quantization.

Or watch Tim Dettmers, who is releasing code to run Mixtral 8x7b in just 4GB of RAM.

link