| "Small" is 32b a9b for 19GB @ Q4_K_XL 20GB @ 100,000 context. But for some reason... LM studio isnt loading it onto gpu for me? I just updated to 0.3.28 and still wont load onto gpu. Switching from Vulkan to rocm. It's now working properly? https://docs.unsloth.ai/new/ibm-granite-4.0 Fantastic work from unsloth folks as usual. As it's running in roo code, it's using more like 26GB of vram. ~30TPS Roo code does not work with it. Kilo code next. It seems to be about 22GB of vram. Kilo code works great. The model however didn't 1 shot my first benchmark. That's pretty bad news for this model given magistral 2509 or apriel 15b are better. Better on pass 2, still no 100% 3rd pass achieved. Im predicting it'll be around 30% on livecodebench. Probably like 15% on aiderpolyglot. Very disappointed in its coding capability. I just found: https://artificialanalysis.ai/models/granite-4-0-h-small 25.1% on livecodebench. Absolutely deserved. 2% terminal bench. 16% on coding index. Completely deserved. |