[layer8]

Server applications don’t spawn threads per request, they use thread pools. The extra context switching due to threads waiting for I/O is negligible in practice for most applications. Asynchronous I/O becomes important when the number of simultaneous requests approaches the number of threads you can have on your system. Many applications don’t come close to that in practice.

There’s a benefit in being able to code the handling of a request in synchronous logic. A case has to be made for the particular application that it would cause performance or resource issues, before opting for asynchronous code that adds more complexity.

[YZF]

Thread pools are another variation on the theme. But if your threads block then your pool saturates and you can't process any more requests. So thread pools still need non-blocking operations to be efficient or you need more threads. If you have thread pools you also need a way of communicating with that pool. Maybe that exists in the framework and you don't worry about it as a developer. If you are managing a pool of threads then there's a fair amount of complexity to deal with.

I totally agree there are applications for which this is overkill and adds complexity. It's just a tool in the toolbox. Video games famously are just a single thread/main loop kind of application.

[acdha]

There’s also a really good operational benefit if you have limits like total RAM, database connections, etc. where being able to reason about resource usage is important. I’ve seen multiple async apps struggle with things like that because async makes it harder to reason about when resources are released.

[tcfhgj]

Could you point out the issue here?

Why does async make it harder to reason about when resources are released?

[acdha]

Basically it’s the non-linear execution flow creating situations which are harder to reason about. Here’s an example I’m trying to help a Node team fix right now: something is blocking the main loop long enough that some of the API calls made in various places are timing out or getting auth errors due to the signature expiring between when the request was prepared and when it is actually dispatched because that’s sporadically tend of seconds instead of milliseconds. Because it’s all async calls, there are hundreds of places which have to be checked whereas if it was threaded this class of error either wouldn’t be possible or would be limited to the same thread or an explicit synchronization primitive for something like a concurrency limit on the number of simultaneous HTTP requests to a given target. Also, the call stack and other context is unhelpful until you put effort into observability for everything because you need to know what happened between hitting await and the exception deep in code which doesn’t share a call stack.

[tcfhgj]

I still don't get it.

The execution flows of individual async tasks are still linear, much like individual threads are linear.

Scheduling (tasks by the async runtime vs threads by the OS), however results in random execution order either way.

If there is a slow resource, both, async tasks as well as threads will pile potentially increasing response times.

Wether async or threads, you can easily put a concurrency limit on resources using e.g. semaphores [1]:

- limit yourself to x connections (either wait or return an error)

- limit the resource to x concurrent usages (either wait until other users leave, or return an error)

Regarding blocking the main loop: with async and non-blocking operations, how would something block the main loop? And why would the main loop being blocked cause API calls being timing out? Is it single threaded?

[1]: https://docs.rs/tokio/latest/tokio/sync/struct.Semaphore.htm...

[acdha]

> The execution flows of individual async tasks are still linear, much like individual threads are linear.

Think about what happens:

1. Request one hits an await in foo()

2. Runtime switches to request two in bar() until it awaits

3. Runtime switches to request three in baaz(), which blocks the loop for a while

4. Request one gets a socket timeout or expired API key

That error in #4 does not tell you anything about #2 or #3, and because execution spreads across everything in that process you have to check everything. If it was a thread, you would either not have the problem at all, it would show up clearly in request three, or you’d have a clear informative failure on a synchronization primitive saying that #3 held a lock for too long.

That makes it harder to control when memory is allocated or released in garbage collected languages, too, because you have to be very careful to trigger gc before doing something which can suspend execution for a while or you’ll get odd patterns when a small but non-zero percentage of those async requests take longer than expected (i.e. load image master, create derivative, send response needs care to release the first two steps before the last or you’ll have weird behavior when a slow client takes 5 minutes to finish transferring that response).

Arguably that’s something you want to do anyway but it dramatically undercuts the simplicity benefits of async code. I’m not saying that we should all give up async but there are definitely some pitfalls which many people stumble into.

[otabdeveloper4]

Because async usually means you've stopped having "call stack" as a useful abstraction.