[iTokio]

TLDR:

- OOM will swap or trigger the OOM killer but Linux might try to reclaim memory from caches before and slow down (using a cgroup limit solved this).

- glibc malloc can easily cause memory fragmentation with many threads (switching to Jemalloc solved this)

[omerban]

A couple of points I would add here:

- You should regulate your process's memory usage and throttle when needed, even at the cost of performance.

- Memory regulation is harder in a multi-process architecture because you can't set a memory limit on the total memory usage of all the processes.