Hacker News new | ask | show | jobs
by cbetz 3240 days ago
This. I've been trying to put together a small home lab xeon e3 system that can theoretically support nvme+gpu+infiniband. It's tough.

Typically you get something like 3 full length PCIe slotson a xeon e3 board (c236 chipset). BUT they can't all run full speed at 16x (16/16/16). You have to then choose between 16/8/8 and 16/16/0 due to the lack of PCI lanes.
1 comments
powercf 3239 days ago
Can you explain why you need 16/16/16? Wikipedia tells me that one x16 PCIe 4.0 connection has a capacity of 32GB/sec (x8 is 16GB/sec). A quick search shows NVMe drives with a sustained read or write speed of ~2-4 GB/sec. Equally for the GPU - you need to move a lot of data in/out of the GPU to saturate a x8 connection.
link
jacquesm 3239 days ago
It's rare to run that sustained, usually it's bursty so the extra bandwidth is very much needed.
link