Hacker News new | ask | show | jobs
by davidpardo 4455 days ago
Being a user of both Linux and PostrgreSQL, I'm very interested in this issue, but I only understand some of the words...

Could everybody wiser than me tell me if I should be concerned and the possible implications of these decisions? Should I invest in alternative platforms?

6 comments

In NUMA front Linux is, as far as I know, leaps and bounds ahead of FreeBSD, Solaris and Windows. All four platforms offer ways to tune process and memory allocations by hand, but Linux is only platform that puts lot of work to make automatic NUMA scheduling actually work. It is actually pretty difficult problem. Something to read if you are interested:

http://queue.acm.org/detail.cfm?id=2513149 http://lwn.net/Articles/591995/ http://lwn.net/Articles/568870/

These are very minor issues and will be noticed by very few.

If you are running a huge database and need every possible bit of performance this will matter, otherwise it's not something to worry about.

You probably don't need to worry about it, it's just further proof that the postgresql devs do it right, and you're in safe hands.
Back in ye olden times when I was an Informix DBA, we had to worry about stuff like this for storage.

It was always a fight with the storage guys, because they wanted to use their fancy Veritas File System for optimizing disk utilization, and us prima-donna DBAs wanted raw LUNs and allow the database engine to manage our disk, because it maximized our transaction throughput. Some DBAs even wanted whole disks allocated, so they could control where data lived from a disk geometry POV. There were (mostly) valid arguments for doing this, most of which have gone away over the years.

This is an issue like my disk issue -- corner cases that need to be thought about in situations where you are investing lots of engineering effort into your databases. If you don't have a couple of angry DBAs whom you're always arguing with, you don't need to worry about this.

It is something that could be helpful to have at the back of your mind if you're tasked with optimizing postgres and OS settings for big workloads.

But it's one of a myriad of little things, not something that could inform a platform decision. It's much more interesting for kernel devs than it is for postgresql users.

I think what this shows is that issues related to interactions between RAM, caches and CPU cores are becoming a lot more complex on all platforms.

same here. Was a FreeBSD user, but find hard to find VM a few years ago that support FreeBSD, may switch back, if this issue is NOT addressed.
First issue is relevant only on systems that have more than one NUMA node, which is probably every meaningful physical server and essentially no VM (at least on Xen, multiprocessor VMs are single NUMA node), as it does not make much sense to advertise NUMA topology to guest VMs.

Second issue is relevant for postgresql mostly only if you use very large shared_buffers which anyway is not recommended for general workloads. Writing page that exists on disk and was not read short time before is not especially common thing to do.

NUMA can absolutely ping you in virtual servers, but without access to the hypervisor you'll never know why it's happening (JVMs straddling NUMA regions have caused me pain in the past, when the guest was split across memory regions).
The point is that kernel inside VM guest knows nothing about NUMA, so it cannot do any kind of NUMA optimizations hence such optimizations cannot hurt performance as they do not happen at all.
Actually, KVM can allow you to create NUMA domains inside the guest, for better or worse.
If you're running in a VM, the VM may impose its own constraints on the disk write and virtual memory issues.