[littledanehren]

Yep, that's basically what we do. User-level threads are called 'coroutines' and they're scheduled cooperatively, so they don't incur the overhead of user-level preemption. Cooperative scheduling is fine for us because we can ensure that a single coroutine won't hog the CPU or issue blocking system calls.

It'd be nice if some pthread implementation gave us what we need, but I don't think they do. They probably are too heavy an abstraction, even if implemented in user space, providing too strong guarantees for context switching and scheduling. For example, we're never going to set individual signal handlers on different coroutines, but pthreads guarantee this capability. So even user-level pthreads which don't support preemption will require a system call on context switches, just as ucontext.h's swapcontext does for similar reasons. But as Slava points out, the performance penalty is something that we need to verify in the future with benchmarks.

What would be really nice is if the OS, or at least a standard application framework usable from C++, supported this. But it doesn't, so we build our own.