[brainscdf]

My personal best practice is to always create a thread pool on program startup and distribute your tasks among the thread pool. I use the same best practice in all other languages too. Is this best practice sound or can it lead to problems in some corner cases?

[hinoki]

There are lots of details that might cause problems:

* Do your tasks block? How many threads do you need to make sure you can use all your CPUs.

* Do your tasks access different sets of memory? Would keeping similar tasks on the same CPUs reduce cache misses.

* Do your tasks have different priorities? You might need a pool for each priority.

For a UI program that isn’t doing anything really intensive or real-time, having a common thread pool makes a lot of sense, and can reduce resource use (stacks add up once you get to many 10s or 100s of threads...), and improve latency (a work queue with many threads will get more CPU than another with the same amount of work but fewer threads)

[londons_explore]

Case in point:

I used nodejs for a project, and assumed that "it's all javascript on one thread" would leave threading issues behind.

My application curiously stopped responding whenever I had 5 or more users. Connected users could continue to do anything, but new users couldn't connect, and existing users sessions would hang when executing any code that wrote to a logfile, making debugging even harder. Using the nodejs debugger, the internals of write(...., cb) were just never calling the done callback.

After hours of head scratching I found that most IO from nodejs is not asynchronous and callback based as the docs suggest, but is in fact blocking IO done from worker threads. My process was using pipes to communicate with other processes, and those pipes were doing blocking writes, and when blocked, the worker thread was blocked.

There are 4 worker threads by default, so whenever 5 users were using the system, all worker threads were tied up and it would fail. It would have been nice for nodejs to at least have printed to the console "All worker threads busy for >1000ms. See nodejs.com/troubleshooting/blockingfileio.htm" or something.

[dirtydroog]

As far as I'm aware, node.js is a wrapper over libuv which is a truly asynchronous socket IO library. It fakes file IO async ops with thread pools because on Linux file IO isn't async at all.

[clarry]

Also:

* Do you have sufficiently large batches that you can efficiently assign to one thread?

If not, then you're just wasting a lot of time waking up to receive inputs, assigning them to threads (-> put them on a work queue or similar, with all the locking / atomics), and waking up a thread to pull an item (locking / atomics), process it, go to sleep...

It's easy to end up spending more time juggling tasks and switching tasks than performing any useful work.

[cjfd]

The thing I would worry about here is that perhaps not all of your tasks have the same performance demands. There may be tasks related to RPC that should run as quickly as possible and tasks related to computation that could take a long time. If all of the threads in the threadpool are busy with an expensive computation there could not be left any to quickly handle RPC requests.

I personally prefer to do as much as possible just in one thread, where you can run things asynchronously with a single threaded message loop and then have a thread pool next to that for expensive computations. This also tends to reduce the number of things that need to be protected with a mutex.