[Tepix]

If you want to inference big LLMs on CPU, you really want those 2x 12 memory channels a Dual EPYC system offers you for decent speeds.

[chipdart]

> (..) you really want those 2x 12 memory channels a Dual EPYC system offers (...)

I had to check and I was amazed that there are companies selling workstations with dual EPYC processors, providing a whopping 256 CPU cores and over 2TB of DDR5. All in a desktop form factor. Amazing.