Just as a reminder:
"So, malloc on Linux only fails if there isn’t enough memory for its control structures. It does not fail if there isn’t enough memory to fulfill the request." - http://scvalex.net/posts/6/
Malloc can also fail if you're out of address space without necessarily being out of memory.
NT does a much better job of separating these concepts than Unix-family operating systems do. Conceptually, setting aside a region of your process's address space and guaranteeing that the OS will be able to serve you a given number of pages are completely different operations. I wish more programs would use MAP_NORESERVE when they want the former without the latter. (I'm looking at you, Java.)
One day, perhaps when I am old and frail, we will achieve sanity and turn overcommit off by default. But we're a long way from being able to do that now.
These days, I describe malloc() as "a function which allocates address space", to avoid confusion. Which means it makes sense that malloc() returns NULL if you are out of address space, even if you have lots of memory. (But so many people don't check malloc()'s return anyway...)
This is a mostly-untrue statement because it makes unreliable assumptions about the host system. It depends on the vm.overcommit_memory setting and the programmer should never make assumptions about why or when malloc might fail. Read more on Rich Felker's excellent blog post here: http://ewontfix.com/3/
Another is if you are allocating lots of memory with alternating mprotect() permissions. On some systems (AIX for example) this uses up all of the memory for control structures WAY before hitting the address space limit (I've seen it fail after just a couple of GB).
NT does a much better job of separating these concepts than Unix-family operating systems do. Conceptually, setting aside a region of your process's address space and guaranteeing that the OS will be able to serve you a given number of pages are completely different operations. I wish more programs would use MAP_NORESERVE when they want the former without the latter. (I'm looking at you, Java.)
One day, perhaps when I am old and frail, we will achieve sanity and turn overcommit off by default. But we're a long way from being able to do that now.