[kent8192]

Oh, boy. This good tool hates my LM Studio... The following message appears when I run Bonsai in my LM Studio. I think my settings have done something wrong. ``` Failed to load the model Error loading model. (Exit code: null). Please check the settings and try loading the model again. ```

[liuliu]

It needs a mlx fork because the lowest bit in mlx is 2 currently (for affine quantization).

[riidom]

That mlx is for apple hardware only, though? Or did I misunderstand something.

[dragonwriter]

It needs a llama.cpp fork, too; so the stock runtime (based on stock llama.cpp) used by LM Studio presumably won't work for it.

[dodos]

Same issue here, wanted to give it a shot but ran into that error trying to load the model in lm studio.