[baq]

TLDR of the LMKL thread: 120GB RAM postgres with hugepages=off, lock contention went from terrible to abysmal. nothing to see here except that amazon for whatever reason runs DB tests with huge pages disabled. (hope I'm not paying for RDS and auroras like that in production!)

[Twirrim]

Huge pages has had a spotty history, that lead to people being paranoid about it, and no doubt a whole bunch of folks just disable it "because that's what we've always done". It has been stable and reliable for quite a while now, would really hope folks could move away from that perspective.

[jeltz]

Are you sure you are not thinking of transparent huge pages? They have a spotty history but you are supposed to run big PostgreSQL instances with huge pages, not transparent huge pages.

[lstodd]

I remember when support for them just appeared and you had to LD_PRELOAD a shim IIRC to make Postgres actually use them we jumped on it, enabled them immediately and got a pretty significant boost, around 15-20%, yes.

That was idk, 2008-9 -ish? I don't know what spotty history you are talking about, if you have multigigabyte address spaces floating on a machine it's stupid not to use hugepages.

[nijave]

I tested it once about 2 years ago on Azure VM and got a nice 10-15% perf boost on pgbench (I want to say at least 64GB shared mem)

[nijave]

In fairness, AWS could (and almost certainly is) using their own kernel build that does who-knows-what

[dist-epoch]

Many people have desktops with 128 GB RAM. Should they enable hugepages? I've never heard this recommendation for a desktop.

[nijave]

Huge pages is good when a single process is reserving a giant block of memory which I think isnt that common.

You might have transparent huge pages on by default depending on the distro

[fc417fc802]

No idea about hugepages but mmaping giant blocks of memory is extremely common. IIRC Chrome maps something like 1 TB on startup.

I think (?) you don't actually see benefit from hugepages unless you actually write to a large amount of memory though.

[baq]

If they’re running any sort of a vm (which they probably do with this amount of ram) they absolutely should and also should consider pre-reserving them.

[andriy_koval]

> nothing to see here except that amazon for whatever reason runs DB tests with huge pages disabled

do you consider huge pages disabled as some discouraged config? If data doesn't fit into memory, it means single lookup will read multiple NVME pages instead of single, which could lead to significant regression.

[mplanchard]

Also was only on ARM, wasn’t it?

[pavon]

I think that ended up being a red herring. It just happened to be the case that the ARM test had huge pages disabled while the AMD64 test had them enabled.