[hinoki]

There are lots of details that might cause problems:

* Do your tasks block? How many threads do you need to make sure you can use all your CPUs.

* Do your tasks access different sets of memory? Would keeping similar tasks on the same CPUs reduce cache misses.

* Do your tasks have different priorities? You might need a pool for each priority.

For a UI program that isn’t doing anything really intensive or real-time, having a common thread pool makes a lot of sense, and can reduce resource use (stacks add up once you get to many 10s or 100s of threads...), and improve latency (a work queue with many threads will get more CPU than another with the same amount of work but fewer threads)

[londons_explore]

Case in point:

I used nodejs for a project, and assumed that "it's all javascript on one thread" would leave threading issues behind.

My application curiously stopped responding whenever I had 5 or more users. Connected users could continue to do anything, but new users couldn't connect, and existing users sessions would hang when executing any code that wrote to a logfile, making debugging even harder. Using the nodejs debugger, the internals of write(...., cb) were just never calling the done callback.

After hours of head scratching I found that most IO from nodejs is not asynchronous and callback based as the docs suggest, but is in fact blocking IO done from worker threads. My process was using pipes to communicate with other processes, and those pipes were doing blocking writes, and when blocked, the worker thread was blocked.

There are 4 worker threads by default, so whenever 5 users were using the system, all worker threads were tied up and it would fail. It would have been nice for nodejs to at least have printed to the console "All worker threads busy for >1000ms. See nodejs.com/troubleshooting/blockingfileio.htm" or something.

[dirtydroog]

As far as I'm aware, node.js is a wrapper over libuv which is a truly asynchronous socket IO library. It fakes file IO async ops with thread pools because on Linux file IO isn't async at all.

[clarry]

Also:

* Do you have sufficiently large batches that you can efficiently assign to one thread?

If not, then you're just wasting a lot of time waking up to receive inputs, assigning them to threads (-> put them on a work queue or similar, with all the locking / atomics), and waking up a thread to pull an item (locking / atomics), process it, go to sleep...

It's easy to end up spending more time juggling tasks and switching tasks than performing any useful work.