Hacker News new | ask | show | jobs
by walrus 3005 days ago
It's not the case that malloc always works under Linux. It may return NULL if the memory is not available. Try it out! Write a program that allocates half your memory, fills it with 0xFF, then sleeps. Run two instances of it. The malloc will fail in the second one.

According to [1], the OOM killer is a consequence of the fork syscall, and can't be removed without breaking backward compatibility.

[1] https://drewdevault.com/2018/01/02/The-case-against-fork.htm...
1 comments
dooglius 3004 days ago
The man page for fork indicates that it can fail and return ENOMEM just fine. Technically, it's only supposed to be returned in special conditions where the kernel doesn't have enough memory, but this seems like much less of a breakage than going around killing random processes.
link
geofft 3004 days ago
It certainly is permitted to fail by spec. The goal of the entire overcommit / OOM killer business is to yield better real-world outcomes.

If I have overcommit off, and a program that allocates more than half of my physical memory (RAM + swap), it can't fork - even if it's going to immediately exec a tiny program. If I have overcommit on, it can fork and exec, and nobody gets OOM killed.

The tradeoff is that, if I have overcommit on and the child process starts modifying every page instead of exec'ing, then the OOM killer triggers. The bet is that a) probably the child process will be killed, since it has the most physical memory in use (assuming the parent process has not touched every page it has allocated), so this isn't worse than preventing the child process in the first place, and b) this situation is rare.

link
twoodfin 3004 days ago
The point is more that processes holding on to a lot of memory often fork and exec and don’t actually need 2X their current allocation in-between.

Yes, there are alternative APIs that avoid the need to temporarily hold on to that memory in both processes, but the idiom is still incredibly common. And it’s not unusual for server-style processing to fork without an exec and largely share their parent’s allocation via copy-on-write. That defers allocation until there’s a page fault on a simple memory write, which doesn’t have a particularly helpful mapping to C if you want to return failure to the copying process.

link