Hacker News new | ask | show | jobs
by gsvelto 1306 days ago
Here's the thing: they're not! The reason those users where crashing was because something, somewhere (possibly in the graphics stack) was reserving ton of space without using it. We had crashes on file with 20+ GiB of free physical memory which was the reason why I started looking into why a machine with so much free memory could suffer a crash.

I hope I described how this whole things work because Windows memory management is not well known and some things about it are counter-intuitive; especially if you're coming from Linux.

1 comments

I can confirm this is the case. Firefox reserves tens of gigabytes and then only actually uses a few. But since those tens of unused gigabytes are still committed, nothing else on the system can use it, and Firefox (+ tons of other apps) will OOM when the total committed memory (of all applications) reaches the total amount of RAM installed.

I have 17 crashes on file (in about:crashes) that are OOMs caused by Firefox committing too much memory and not using it. My solution is to restart the browser when committed memory reaches 38GB or so, but that clears all private windows which annoys me. I shouldn't have to restart the browser so often.

Could you open a bug on our tracker, and point to those crash reports? I'm very interested in analyzing real-world scenarios where this happens to figure out where those committed areas are coming from. Our best guess is that they come from the graphics stack because we account all the memory we commit in Firefox and there's nothing missing, so wherever this is coming from it's outside of our direct control.
I actually did open up about:memory a few hours ago and now, and it looks like some growth is coming from "heap-unclassified" in the main process.

As for opening a bug on your tracker, meh but I can give you a list of report IDs:

    bp-e6392a4b-44e8-4013-a80e-1027b0221109  
    bp-dae0a056-3c6a-43ea-ad3f-5fe1a0221006  
    bp-0be1e0f7-0eca-4744-a05c-ce2030220927  
    bp-65c63eea-fb45-419d-a58a-82ef20220915  
    bp-061b6131-5ae3-4de1-a7ba-0c4950220830  
    bp-32f879a0-f76f-4a5b-bef9-f013f0220820
From the looks of it you have a minuscule page file (16MiB), that's why you're running out of commit space. I suggest setting it to auto-resize (the Windows default) which should provide a much better experience even with other applications, unless you need to keep it small because of storage concerns. Generally speaking Windows behaves very poorly with small swap files because of its inability to overcommit memory.
Firefox should not be relying on overcommit in the first place. The issue isn't that I "only" have 40GB of commit space - it's that Firefox keeps taking more and more to the point of crashing (itself and everything else I have running).

And yes, my page file had grown to 64GiB before, when I stupidly left it on automatic. There's a reason it's at 16MiB now - it's because completely turning it off disables memory compression. Not like that helps much due to the lack of overcommit.

You're assuming that Firefox is eating all your commit space, but it's not, far from it, I can see it from the crash reports. You've got other applications running and they are also fighting for it, leaving large chunks unused. Without some swap space available Windows is leaving 10+ GiB of physical memory free which it cannot use for anything else.

This is how Windows behaves by design, there's nothing we can do about it. Windows without a swap file sucks.