[seriesf]

I wish there was a more robust market for external nvme-in-box peripherals. Having to deal with vendor brain damage is never fun, and nvme devices are so small and draw little enough power that they’re really the perfect external peripheral.

[ComputerGuru]

Do you mean something like an external enclosure containing a backplane connecting to NVMe drives, connected to the main server over SAS 12gbps, or something else entirely?

[derefr]

Connected to the main server over (perhaps multiple) Thunderbolt cables, I'd assume. Since you really do just want PCI-e passthrough.

[seriesf]

Yeah the problem is you want more lanes than thunderbolt brings. Two nvme devices, even cheap ones, can swamp an x4 pci slot. There are various connectors designed to bring faster PCI buses out of the box, but no real standard.

[derefr]

There's no reason that Thunderbolt (especially optical Thunderbolt) can't be that standard. In fact, there's no reason that Thunderbolt controllers need to live in the CPU, rather than living on PCi-e HBA cards (like InfiniBand or RAID controllers do.)

It just hasn't happened yet because of a lack of enterprise demand. The pendulum hasn't swung back to all-externalizing blade servers quite yet.

[namibj]

Actually they currently just put a retimer chip on the backplane, attach the NVMe drives to the otherwise passive (as far as high-speed data goes) backplane, and use QSFP+ twinax cabling between the backplane and a sometimes passive "HBA" card that adapts from PCIe slot to this QSFP+ cage. Afaik they can run PCIe3 x4 per cable. If you look for open compute hardware, you'll find various modular blade-like designs.

[JackRabbitSlim]

Ironically, a big 16x SAS port is about the only external interface fast enough for it.