[Der_Einzige]

Any data on inference speed? I’ve found that the non quantized model was much faster on GPU than the quantized versions due to lower GPU utilization.

[api]

It's a RAM tradeoff. If you have enough GPU RAM to load the non-quantized model it may be faster.