[amag]

Actually there is one reason for malloc to return NULL even with virtual memory, your process can run out of address space.

[roganp]

Or run out of room in the paging file. Your addressable memory cannot be larger than physical memory without a backing store.

[prodigal_erik]

Depending on VM_OVERCOMMIT_MEMORY, Linux might give out address space well beyond the size of the pagefile, hoping many of those pages are never written to (e.g., most threads never get anywhere near the bottom of their default stacks).