[zone411]

Yes, that's very fast. The same query on Groq, which is known for its fast AI inference, got 249 tokens/s, and 25 tokens/s on Together.ai. However, it's unclear what (if any) quantization was used and it's just a spot check, not a true benchmark.

https://www.zdnet.com/article/cerebras-did-not-spend-one-min...

[Tetraslam]

Met them at an MIT event last week, they don't quantize any models.