[aurareturn]

If it's 150 t/s, that's barely faster than Nvidia GPUs who are batching a lot more and are a lot more cost effective. Add in the Groq piece and Nvidia claims it can do 400 tokens/s.

[eli]

I assume it’s just oversubscribed. I’m sure it “can” go faster. But yeah that was my point.