[conradev]

In-memory compression means the memory is inherently dirty memory

On Apple platforms if you mmap a read-only file into the process address space, then it is "clean" memory. It is clean because the kernel can drop it at any time because it already exists on disk. You essentially can offload the memory management to the kernel page cache.

The downside is that if you run up to the limit and the "working set" can't fit entirely in memory, then you run into page faults which incur an I/O cost.

The advantage is that the kernel will drop the page cache before it considers killing your process to reclaim memory.

That said, I don't know the typical access patterns for neural network inference, so I don't know how the page faults would effect performance