[ZiiS]

Most of the ram is for storing the model once it is loaded it is read only so will not harm an SSD.

[Incipient]

It only reads from memory,not swap directly. If it needs to read something from swap, it'll write out something from memory to swap, then read the swap into memory. Reading 1gb of swap, will essentially write 1gb to the ssd too. (rough numbers)

Correct me if I misunderstand swap?

[mihaic]

If the underlying data hasn't changed, the page isn't written to disk. CPUs do keep track of writes that mark a memory page as "dirty".

[simcop2387]

That's basically right. I'm not sure if Linux or windows will keep track of the pages it read out of swap to know if they're still there and valid, but there's a better way for this that I think at least ggml supports where it stores a copy of the model unpacked and ready on disk as a cache for doing the work rather than relying on the OS virtual memory to handle it. This should be faster than the OS VMM (though probably not by much) but since it'll know which pieces it needs to leave on disk and where they are it should be much safer as far as writes go since it will know enough to not write multiple times like that.