[int_19h]

For text inference, what you want is M1/M2 Ultra with its 800 Gb/s RAM. Max only goes up to 400 Gb/s.

[ls612]

Yeah but the ultra only goes in desktop platforms which may be limiting to some.

[int_19h]

But that's no different from mid-to-high-end GPUs, which is what the original ask was about.