[codedokode]

Please note that 8 x 16 Gb DDR4 modules cost $480, and have a bandwidth of 200 Gb/s, the only problem you need 8 DRAM controllers.

[nicce]

> the only problem you need 8 DRAM controllers.

Which is a huge problem? Even using 2x memory controllers in typical consumer motherboard can make system very unstable.

[codedokode]

Why is that so?

[jaxytee]

Running large MOE models won't saturate the EPYC memory.

You'll never see that 200 Gb/s running inference.

[edg5000]

Because it'll bottleneck at the CPU? Or because 200 bB/s is overkill when running a model (that fits in memory)?