[treprinum]

That's been done already. See DeepSpeed ZeRO NVMe offload:

https://arxiv.org/abs/2101.06840