Y
Hacker News
new
|
ask
|
show
|
jobs
by
Aurornis
84 days ago
I should clarify that I'm referring generically to the types of quantizations used in local LLM inference, including those from Unsloth.
Nobody actually quantizes every layer to Q4 in a Q4 quant.