[the_mitsuhiko]

In a language with good threading support, internal threads are typically making things easier rather than harder. I tend to spawn lots of threads in Rust just because I can and it simplifies the code a lot over having to do some async callback mess.

In particular there is no sane way to async waitpid() on POSIX.

[geofft]

pselect / self-pipe / etc. on SIGCHLD, and on receipt of SIGCHLD, loop through all known child processes with wait(WNOHANG).

... I'm not sure I can disagree with "no sane way".

[the_mitsuhiko]

The only thing you can do from a signal handler is flipping a static and only one signal handler can be set. Good luck making this reusable.

In fact, I challenge you to solve the problem "spawn process; waitpid for 15 seconds; otherwise kill hard" in Rust (or C++ if you feel like) on POSIX once with threads and once without threads by sticking to what's permitted in the standard and so that multiple processes can be waited for.

Then also measure CPU impact :)

[gpderetta]

> The only thing you can do from a signal handler is flipping a static and only one signal handler can be set.

This is false, you can call any async signal safe function. Incidentally write is one of them.

Another trick is the close-on-exit pipe.

[geofft]

"Async-signal-safe" is a C concept (from the POSIX world where C is your interface to the system, and library calls vs. system calls are behind the abstraction layer), so it doesn't directly apply to Rust. But the underlying semantics of signals are simple to describe: you get interrupted at some instruction pointer and jump into a new function. You can do whatever you want provided you uphold safety, correctness, liveness, etc.

If you change a variable, it has to be one that isn't prone to being cached in a register or the stack by the main program. POSIX's sig_atomic_t does this; in Rust you can use the normal atomic types. They are a tiny bit too careful if this is thread-local, but an ordinary thread-local variable is permitted to be cached within the same thread, and signal handlers break that.

If you take a lock, you have to do something reasonable if the lock is already held, including by the code you interrupted. So you probably shouldn't lock at all. The biggest reason for a POSIX function not to be async-signal-safe is because it wants to call malloc, which takes out a lock (at least a per-thread or per-CPU lock) on the heap. If you get signaled during a malloc, and the signal handler tries to malloc, you deadlock.

But anything that does not risk liveness or correctness problems is fair game. In particular, basically all system calls are fair game, since they're just sending a message to the kernel. C's fprintf() will want to buffer in userspace, which involves an allocation, but write() will at most buffer in the kernel, and the kernel-side code doesn't have the problem of having flow control interrupted while you're in a signal handler. Even if you were previously in a blocking write() when you received a signal, the kernel will return from its implementation of write before delivering the signal back to userspace, so there isn't a re-entrant call to the kernel-side write code. libc's fprintf() doesn't have that luxury.

(And yes, the concept of Rust on POSIX is a bit ill-defined, because POSIX is a set of C-language APIs, which can be implemented in any valid way in C, including header macros. Rust threads use pthreads, yes, but inter-thread communication doesn't involve whatever sig_atomic_t is typedef'd or #defined to.)

[gpderetta]

Note that the lock is a red herring. A single threaded malloc wouldn't be async signal safe either unless explicitly designed to be so which is hard. In fact anything that touches mutable state, including thread local state is a problem.

[geofft]

Yeah, good point. The weird part is that it almost introduces a new thread, in that it introduces a new flow of control (and, if you ask for it, a new stack), but it doesn't count as a new thread in all the usual ways, like thread-local variables or PIDs. So any code called from a signal handler must be "thread-safe", but that thread-safety cannot rely on assigning an identifier to the thread, because you're using the same identifier as the parent thread.

[the_mitsuhiko]

> This is false, you can call any async signal safe function. Incidentally write is one of them.

Which malloc() is not. Anything that might internally allocate is out of the question. The list of functions that are safe to call in C alone is very limited and even then the question of errno arises.

[gpderetta]

why are you bringing up malloc?

[the_mitsuhiko]

Because you do not necessarily know which call will allocate memory in Rust and even in C it can be tricky. You literally can only go with the whitelisted functions.

[geofft]

Here's a sound implementation with signals using only the Rust standard library and POSIX, no threads:

https://play.rust-lang.org/?gist=ba4802a59f462cb8bce0c1bac92...

It would be significantly simpler with sigwaitinfo() if you didn't care to do a real select loop, but I assume most programs want a real select (or poll/epoll/whatever) loop.

[the_mitsuhiko]

Which proves my point. It's messy, inefficient and now you wrote code which is not composable as you have a global (and the only) signal handler.

[geofft]

I don't think it's less efficient than threads (but someone should test it!). It scales better.

It's not composable, but it's not impossible to work around that if you can assume non-POSIX. Platforms with kqueue just get this right because kqueue can wait for processes. Linux will let you use a different signal to alert for process completion, so pick one of the realtime signals (which is its own game of global namespaces, but hey), and library code that uses SIGCHLD won't be affected. So that's Linux, OS X, and the BSDs. I don't know of a good workaround for Solaris.

A POSIX-compliant way that makes this more composable is to make a single, dummy process to be a process group, setpgid() all your actual children into that process group, and have that process spend its time doing waitpid(0, WNOHANG) and sending you notifications over a pipe or something. That is higher overhead, but my guess is that scales better for many child processes (O(1) extra processes vs. O(n) extra threads).

[the_mitsuhiko]

It's less efficient if the "master" has to select over multiple things. Eg: waiting for children to finish, do some IO with them etc.

In that case you are pretty much limited to polling on the thing (unless apparently you use kqueue, did not know you can wait on process events).