[saas_co_de]

I think this is correct.

With 24X NVMe servers now common and cheap (AMD EPYC) you can get 96GB/s (theoretical max, and real is not far off) onto NVMe which is same order of magnitude as memory bandwidth.

When PCI-e 4 comes out next year the bandwidth will double and SSD manufacturers will have no problem doubling the performance of NVMe devices once the interface supports it so at that point the bandwidth on NVMe will surpass RAM.

The major caveat is that sequential read/write is much faster than random (this can be offset to a certain extent if you can queue up many random requests at the same time but each random request has a very high latency) so I would expect that switching from 4K to 2MB pages would have significant advantages on the storage side beyond efficiencies gained in the kernel.