| I wrote a networking daemon (SMTP greylisting proxy; now unmaintained, but <plug> http://spey.sf.net </plug>) using coroutines once. Non-preemptive multitasking made reasoning about the problem way easier, because I had no concurrency to worry about, while still allowing inversion of control flow which meant that the SMTP state machines were small and simple. It worked really well. ...until I started getting bug reports from users about weird irreproducable crashes. After a very, very long time I eventually figured out that if spey was built on a system where sqlite had been compiled with pthread support, then it automatically linked in a thread-safe malloc, which tried to fetch the current thread ID, which on some versions of pthreads on some Linux architectures on some glibc versions with some kernel versions, was stored at the top of the stack. It used alignment tricks to find this. So, every time something called malloc(), the libc was doing the equivalent of: threadid = *(int*) (alignup(sp, 16MB) - sizeof(int))
Except my stacks weren't allocated through pthreads, so threadid ended up being garbage. Hilarity ensued.I eventually rebuild the coroutine library to be a wrapper around pthreads with a big lock to ensure that only one would run at a time. Which was kind of evil, but much more reliable. (Previously I was using my own coroutine implementation based on getcontext/setcontext.) So, uh. I can't remember if I had a point any more; but the pain remains. |