But then we end up with the OOM Killer, which is awful - randomly kill the biggest process because "reasons".... It would be better if the OS could say no.
In this case, the OOM Killer would be better, because a parent process is allowed to sacrifice one of its children, so the browser could kill the least-recently-used tab instead of itself.
If allocation predictably fails, you don't need an OS-level OOM killer to kill least-recently-used - you could just do said killing manually on failed allocation yourself. And you'd be able to do so in a much more controlled manner too while at it, instead of hoping the OS does what you want. (and if an OS/stdlib wanted to, such behavior could be made the default operation on allocation failure)
Right, it wouldn't help when one process wants more memory but you want an unrelated one to get killed, but the question here was about a browser killing one of its own tabs instead of the main browser process dying. (though, for what its worth, in the case where processes themselves can't decide how to free memory, I, as the user, would much prefer to be given the option of what to kill anyway; linux completely fails to do that, and given that overcommitting affects DEs too, it'd be pretty complicated to allow for such a thing)
not dynamically chosen though, at least in the case of earlyoom; whether to prefer killing the browser, or a random long-running process that has built up a couple gigabytes of RAM usage (or even just a bunch of small processes I don't need) will entirely depend on the intent (or lack thereof) behind the process, and what's currently happening in the browser.
Without over-committing, you could be preventing something that would work anyway. With it, the OOM could (often does) pick the biggest process that could very be using its memory legitimately AND be the most important process of this machine too.
https://unix.stackexchange.com/questions/282155/what-is-the-...