[palmer_fox]

Perhaps the wrong thread to ask this question... Is it not possible to load a model on something like an NVMe M.2 drive instead of RAM? It's slower of course, but only 5-10x if I understand correctly.

[kirill5pol]

Yes but they’re slow enough on normal hardware for that 5-10x to be painful…

[mirekrusin]

Can you RAID them?

[brucethemoose2]

Technically yes?

But its way beyond the point where its going to help LLMs. CPU RAM is already "too slow" in machines big enough for multiple NVMe SSDs.