Y
Hacker News
new
|
ask
|
show
|
jobs
by
aarnphm
1100 days ago
Hi there, 8bit and 4bit is currently supported on main. GPTQ is working in progress, as well as GGML
1 comments
nacs
1100 days ago
GPTQ support would be amazing (AutoGPTQ is an easy way to integrate GPTQ support - it's basically just importing autogptq and switching out 1 line in the model loading code).
link