[pjc50]

Someone familiar with this please confirm: if you have a node.js app, which is true single-threaded, do people run multiple copies per physical CPU, or do they just max out one core and leave the rest idle? Or lease the cores separately from their hosting provider by running a bunch of one-core container instances or something?

[gbuk2013]

Node supports 2 ways of getting an additional event loop threads (this being the “single-threaded” part that people talk about with Node often without understanding much about its internals, as the Node process itself spawns many threads in the background).

The first mode is child process: the main process forks an entirely separate instance of Node with its own event loop, which you communicate with over IPC or some network socket.

The second mode (introduced fairly recently) is the ability to spin off worker threads which have their own event loop but share the worker thread pool of the main process. I think there is a way to share memory between these threads via some special type of buffer, but I have never used them.

The first mode maps directly to the idea of micro-services, just running on the same machine. This is why it is not really used AFAIK in modern cloud based apps, with single core micro service instances used instead. This approach has a higher latency cost but allows cheaper instances and much simpler services - it very much depends on the use case to decide if that is correct choice or not.

[Jarwain]

At my company we use node's cluster module, configured to effectively run a child process for each physical CPU on our Azure App Service.

We have a load balancer in front of that to scale app service instances.

[jiggawatts]

I mentally translate “modern web development” to: proxies for the proxies and layers of load balancers upon load balancers to make the envoys work with the ingress, all through an API management layer for good measure… and then a CDN.

“Why is our app so slow?”

“It’s a mystery. Just scale it out some more!”