Hacker News new | ask | show | jobs
by coconut08 702 days ago
how much vram do you need for 4-bit llama 405?
1 comments
zargon 702 days ago
405 billion * 4 bits = approximately 200 GB. Plus extra for the amount of context you want.
link