Go creates the illusion of preemptive multithreading by having implicit safe-points for cooperative multithreading. Each IO operation is such a safe-point. If you write an infinite loop like `for {}` where there are no IO operations in loop body, it will block indefinitely. This will prevent the underlying OS thread from being available to other goroutines. The same thing can happen even if you do have IO operations in there, but the work being performed is dominated by CPU time instead of IO.
An “OS thread” is the most basic part of the program that can actually do things. When it is blocked, it can’t do anything.
OS threads can be expensive, so libraries try to only create a few OS threads.
A “hot loop” (usually, I think, called a “tight loop”) is a loop that does a lot of work for a relatively large amount of time all at once. Things like “looping through a list of every building an Manhattan and getting the average price over two decades.”
With some code like networking, you end up having to wait on other parts of the computer besides the CPU often, for things like uploads and downloads.
“Asynchronous programming” tried to make it easy to keep the CPU busy doing helpful things, even while some of it’s jobs are stuck waiting on uploads/downloads/whatever. This keeps the program efficient, because it can do a little bit of many tasks at once instead of having to complete each task entirely before moving on to the next.
The problem comes when you have a tight loop in a thread that is mostly expecting to be doing asynchronous work around I/O or networking. It is basically trying to juggle with one hand. The program can’t multitask as well, and you end up having to wait longer before it can start each piece of work you want.
Hot loops are often "expensive", or where your program is spending a lot of its time.
If you're doing a hot loop, which may be synchronous, you will block the event loop attached to that OS thread, because a hot loop is presumably not yielding control until it is done.
The problem is that the async model is a form of cooperative multithreading, so if one computation runs for a long time without returning to the main event loop, it can increase the latency for responses to other events. E.g., if one HTTP request takes a long time to process, and many of the worker-pool OS threads are handling such a request, response time goes up for all the other requests. OS-level concurrency is preemptive (timesliced), so one busy thread doesn't block other requests, but of course with much higher overhead in other ways. Best practice is usually to keep event handlers on event-loop threads short and push heavy computations to other OS-level worker threads.
ah, that make sense, I think another user point out that spawning n goroutine does not actually spawn n physical threads but rather queue n task to m thread in the pool, so if we exhaust m thread, n-m task will be blocked.
Thanks for the explanation
I wonder what is the point where each trade off make sense (ie. what is consider heavy computation vs light computation, it probably is related to OS thread allocation time)