Hacker News new | ask | show | jobs
Towards Optimal LLM Quantization (picovoice.ai)
18 points by bejager 751 days ago
5 comments
aa6864aa 751 days ago
How does it compare with AWQ, SqueezeLLM, or newer quantization methods?
link
abcd98 751 days ago
How do you integrate with vLLM?
link
dynamix 751 days ago
Is there a way for me to compress a custom fine-tuned model of my own?
link
bejager 751 days ago
not yet but it's something we have in mind as a future feature.
link
eonlav 751 days ago
Decent platform support - any plans for a Rust SDK?
link
bejager 751 days ago
We continuously work on expanding SDK support, Rust is also on the list.
link
aviel 751 days ago
Any benchmarks with Falcon 2?
link
bejager 751 days ago
we don't support Falcon 2 yet but new models are always on our radar to be added to the platform.
link