[sgt]

So the M3 Ultra is amazing value then. And from what I could tell, an equivalent AMD Epyc would still be so constrained that we're talking 4-5 tokens/s. Is this a fair assumption?

[adgjlsfhk1]

No. The advantage of Epic is you get 12 channels of ram so it should be ~6x faster than a consumer cpu.

[sgt]

I realize that but apparently people are still getting very low tokens/sec on Epyc. Why is that? I don't get it, as on paper it should be fast.

[Aeolun]

The Epyc would only set you back $2000 though, so it’s only a slightly worse price/return.