[jamesaguilar]

I think he has some iffy premises. I'm not certain, but I guess pthread mutexes probably use userspace mutexes under the covers. If they don't you could select a library that does. Grabbing uncontended mutexes should be basically free. It is definitely not a good idea to write all your own abstractions, as he suggests, unless you have a really great understanding of the issues involved. You'd probably still get it wrong sometimes even with that.

Affinity is definitely valuable, but I don't think you should need to disable interrupts for most applications. I'm not even sure if it is generally possible, since interrupts are sometimes used to swap pages or notify the kernel that a page isn't mapped or that a blocked resource is now available. The reason affinity is valuable is not because of kernel interactions. It's because of NUMA and cpu cache swapping. Affinity can prevent thread migration, which is expensive mainly because data also has to be migrated or else accessed in a less efficient manner. Likewise, make sure that if you dispatch an asynchronous call, the handler runs on the same core you sent the call from.

Finally, it's a common fallacy in these kinds of posts to act as if threads can't be used to do shared-little or shared-nothing-style multi-programming. They often aren't, but there's nothing that prevents it.

[javert]

I'm not certain, but I guess pthread mutexes probably use userspace mutexes under the covers.

Correct. man 7 pthreads states:

In NPTL, thread synchronization primitives (mutexes, thread joining, and so on) are implemented using the Linux futex(2) system call.

A futex is a mutex that only does a system call if there is contention; you can google it.

[scott_s]

"Fuss, Futexes and Furwocks: Fast Userlevel Locking in Linux": http://kernel.org/doc/ols/2002/ols2002-pages-479-495.pdf

[robertgraham]

My blog post was quite clear that I'm talking about futexes, and then when it fails to get a lock that it goes into the kernel. Seriously, that's how everyone did it from Solaris to Windows before Linux "invented" the concept AND it's been in Linux for a decade. When somebody says "mutex", you have to assume they already mean "futex".

[javert]

If you fail to get the lock, then you have to wait on the lock, so you might as well go into the kernel anyway. That's the reason futexes do that, instead of just letting the thread spin.

True, there are scenarios when you might prefer to spin, but they're pretty specialized.

My blog post was quite clear that I'm talking about futexes

No, you never mention the word "futex."

When somebody says "mutex", you have to assume they already mean "futex".

I don't think that's a valid generalization at all.

[robertgraham]

The entire point of the post was talk about "lock-free" algorithms where two cores can make forward progress without either having to wait or spin.

What systems have mutexes that aren't built like futexes?

[javert]

Kernels, and probably any kind of parallel user-level runtime like Erlang.

[jamesaguilar]

> I don't think that's a valid generalization at all.

Especially given he was complaining about syscall costs . . . which are nonexistent with futexes . . .

[robertgraham]

They are NOT non-existent, they happen a lot in contention. That's why code fails to scale: as you add CPUs, contentions happen a lot more often, and the number of syscalls shoot through the roof.

[jamesaguilar]

But the cost is not the syscalls . . . it's all those threads getting stopped, swapping memory in and out of the cache, etc. The syscall cost is minor by comparison.

At any rate, you've definitely overstated the case against mutexes in this post. At most, the advice should have been "avoid designs that will suffer from a lot of contention" like locking with lots of shared mutable state. It really has nothing to do with threads-vs-processes as you've made it out to do.

[jamesaguilar]

If there is contention, I think you mean, right?

[javert]

Yeah, edited to fix it. Just a typo.