[focusgroup0]

The fact that Apple didn't ship this in years after Siri acquisition is an indictment of its Product leadership

[sanchitmonga22]

Apple has the silicon, the frameworks (MLX, CoreML), and the models. The gap is putting it all together into a fast, unified on-device pipeline. That's what we're focused on, and honestly, we think Apple will eventually ship something similar natively. Until then, we're trying to show whats possible today on their hardware.

[liuliu]

This is not different from mlx-lm other than it uses a closed-source inference engine.

[sanchitmonga22]

Respectfully, the benchmarks show it is different.

MetalRT and mlx-lm use the exact same model files, identical 4-bit MLX weights. That makes it a pure engine-to-engine comparison:

LLM decode: MetalRT is 1.10-1.19x faster across all models tested

STT: 70s audio in 101ms vs 463ms (4.6x faster)

TTS: 178ms vs 493ms (2.8x faster)

mlx-lm is a general-purpose array computation framework that also supports inference. MetalRT is purpose-built for inference only. That focus is where the performance gap comes from.

You can reproduce these numbers yourself: rcli bench runs the same benchmarks we published. Full methodology: https://www.runanywhere.ai/blog/metalrt-fastest-llm-decode-e...

Yes, MetalRT is closed-source. We're transparent about that. The performance difference is the reason it exists.