Hacker News new | ask | show | jobs
by Bumblonono 703 days ago
It fits a 4090. Nvidia lists the models and therefore i assume 24gig is min
1 comments
michaelt 703 days ago
A 4090 will just narrowly fit a 34B parameter model at 4-bit quantisation.

A 12B model will run on a 4090 with plenty room to spare, even with 8-bit quantisation.

link