Hacker News new | ask | show | jobs
by explosion-s 134 days ago
Just curious, is there any smaller version of this model capable of running on edge devices? Even my Mac M1 with 8gb ram couldn't run the C version.
3 comments
guskel 134 days ago
This semi-quantized version targets the Jetson Orin Nano, but only comes with a simple inference engine.

https://huggingface.co/Teaspoon-AI/Voxtral-Mini-4B-INT4-Jets...

link
sofixa 134 days ago
https://kyutai.org/stt has an implementation for MLX and mentions iPhones, so it should work on edge devices, Macs and iPhones.
link
adefa 132 days ago
I'm curious to see if you are able to run the model now from the CLI?
link