[opportune]

Yes, but thread pools kind of suck for workloads with multiple i/o operations per "run", because they end up with either coloring problems, blocking ineffeciently, or doing extra context switching. Also, most developers set the pool size to the hardware parallelization limit which worsens either the context switching cost or the ability to be smart about synchronization.

At the bare minimum, you want your userspace scheduling to include synchronization tools in addition to a threading abstraction - you have more information available than the kernel about which threads are blocking which others, so you can be smart about starting threads you know are unblocked and not leaving hardware idle while there’s work to be done.

Regarding load, a lot of people use infrastructure tools to limit that these days before the load hits their application - without that, your gripe is not actually a solvable problem in the application itself. You’ll incur an inherent amount of I/O and scheduling thrashing for each ingress event which is completely unavoidable without dropping requests - even with fixed concurrency in your application code you’ll get this. However, as seen in the blog, the problem is not actually with internal application concurrency at all - Golang can handle 1mm active goroutines no sweat.

The problem instead is your hardware simple can’t perform HTTPS, or god forbid json, deserialization fast enough to handle the throughput getting thrown at you, which is distinct from the number of active requests. If you are gonna make a very long running call to some other service and otherwise do nothing in that goroutine while doing so, you can actually handle absurd levels of concurrency, easily in the hundreds and probably thousands or tens of thousands. But you still can’t handle 8 virtual cores trying to deserialize 100 json bodies every 10ms