Hacker News new | ask | show | jobs
by brucethemoose2 1092 days ago
There is even some untapped headroom, as they quantized to Q4 instead of using K-quant.

Not to speak of the potential hooked up to a vector db, swapping out LORAs for different languages and such.