[illumin8]

Short and overly simple answer: If a program allocates 2GB of RAM, it might not need to use it all right away. The kernel pretends like it just gave the program 2GB of RAM, but it doesn't really need to flush all those unused pages of cache yet. It can still use them for other things like disk cache, and when the program tries to access those pages then the kernel will dynamically page them in and out as needed. This explains why a program might have 2GB virtual but only 128MB is resident in actual physical memory pages.

This is overly simple and there are a lot of nuances such as shared memory segments, etc.

[adavies42]

then there's over-commit, e.g. "you want to malloc 200% of the ram? here you go! enjoy! oh wait, you wrote to the second half? have fun finding the power button, sucker!"

can someone explain to me why so many distros, including "enterprise" server stuff, ship with /proc/sys/vm/overcommit_memory set to 1?

[aliguori]

Most software cannot cope with malloc failing. A little swapping is at least recoverable. A failed malloc almost guarantees that your application will crash--sometimes in a very unpleasant way.

[adavies42]

ime the main alternative is the box crashing, frequently without leaving behind enough information to know what went wrong. at least if the app crashes you have a pretty good idea who was incompetent.