[dalias]

No, OOM killer is completely orthogonal to this and is a consequence of not doing correct commit accounting. With strict commit accounting turned on (vm.overcommit_memory=2) fork will correctly fail when there is not sufficient physical backing. But you're correct that fork+exec is a bad model.

[ddevault]

The OOM killer exists because of memory overcommitment, which exists because of fork/exec. The justification for overcommitment is that a big-ass process might fork just to do an exec immediately afterwards. If that is the case, then it would be lame to error out the fork because there's not enough room for a second copy of it. But if it doesn't actually exec, then suddenly there's not actually as much memory as it thinks it has as the forked process starts modifying things. This justified overcommitment in the first place and then it snowballed from there.

[dalias]

No, fork is only one path that can lead to overcommit. Allocation of new memory as COW references to a zero page, and COW writable MAP_PRIVATE mappings of files (such as the writable LOAD segments of any executable or library file) also lead to overcommit unless you do proper commit accounting. Any system that does not need to do detailed commit accounting to avoid overcommit is basically wasting the fact that it has virtual memory/MMU.

[Sir_Cmpwn]

I never meant to suggest that fork is the only path that leads to overcommitment, but that fork/exec generally insists on overcommitment for reasonable use.

[wfunction]

Thanks for sharing this, I wasn't aware of the relationship between the two.