Exactly. People are too afraid of using threads these days for some perceived cargo-cult scalability reasons. My rule of thumb is just to use threads if the total number of threads per process won't exceed 1000.
(This is assuming you are already switching to communicating using channels or similar abstraction.)
The performance overhead of threads is largely unrelated to how many you have. The thing being minimized with async code is the rate at which you switch between them, because those context switches are expensive. On modern systems there are many common cases where the CPU time required to do the work between a pair of potentially blocking calls is much less than the CPU time required to yield when a blocking call occurs. Consequently, most of your CPU time is spent yielding to another thread. In good async designs, almost no CPU time is spent yielding. Channels will help batch up communication but you still have to context switch to read those channels. This is where thread-per-core software architectures came from; they use channels but they never context switch.
Any software that does a lot of fine-grained concurrent I/O has this issue. Database engines have been fighting this for many years, since they can pervasively block both on I/O and locking for data model concurrency control.
The cost of context switching in "async" code is very rarely smaller than the cost of switching OS threads. (Exception is when you'ree using a GC language with some sort of global lock.)
"Async" in native code is cargo cult, unless you're trying to run on bare metal without OS support.
The cost of switching goroutines, rust Futures, Zig async Frames, or fibers/userspace-tasks in general is on the other of a few nano-seconds whereas it's in the micro-second range for OS threads. This allows you to spawn tons of tasks and have them all communicate with each other very quickly (write to queue; push receiver to scheduler runqueue; switch out sender; switch to receiver) whereas doing so with OS threads would never scale (write to queue; syscall to wake receiver; syscall to switch out sender). Any highly concurrent application (think games, simulations, net services) uses userspace/custom task scheduling for similar reasons.
Nodejs is inherently asynchronous and the JavaScript developers bragged during its peak years how it was faster than Java for webservers despite only using one core because a classic JEE servlet container launches a new thread per request. Even if you don't count this as "context switch" and go for a thread pool you are deluding yourself because a thread pool is applying the principles of async with the caveat that tasks you send to the thread pool are not allowed to create tasks of their own.
There is a reason why so many developers have chosen to do application level scheduling: No operating system has exposed viable async primitives to build this on the OS level. OS threads suck so everyone reinvents the wheel. See Java's "virtual threads", Go's goroutines, Erlang's processes, NodeJS async.
You don't seem to be aware what a context switch on an application level is. It is often as simple as a function call. There is no way that returning to the OS, running a generic scheduler that is supposed to deal with any possible application workload that needs to store all the registers and possibly flush the TLB if the OS makes the mistake of executing a different process first and then restore all the registers can be faster than simply calling the next function in the same address space.
Developers of these systems brag about how you can have millions of tasks active at the same time without breaking any sweat.
The challenge is that async colors functions and many of the popular crates will force you to be async, so it isn't always a choice depending on which crates you need.
Please excuse my ignorance, I haven't done a ton of async Rust programming - but if you're trying to call async Rust from sync Rust, can you not just create a task, have that task push a value through a mpsc channel, shove the task on the executor, and wait for the value to be returned? Is the concern that control over the execution of the task is too coarse grained?
Yes, you can do that. You can use `block_on` to convert an async Future into a synchronous blocking call. So it is entirely possible to convert from the async world back into the sync world.
But you have to pull in an async runtime to do it. So library authors either have to force everyone to pull in an async runtime or write two versions of their code (sync and async).
There are ways to call both from both for sure, but my point is if you don't want any async in your code at all...that often isn't a choice if you want to use the popular web frameworks for example.
I can't both perform blocking I/O and wait for a cancellation signal from another thread. So I need to use poll(), and async is a nice interface to that.
Async and GUI threads are different concepts. Of course most GUIs have an event loop which can be used as a form of async, but with async you do your calculations in the main thread, while with GUIs you typically spin your calculations off to a different thread.
Most often when doing async you have a small number of tasks repeated many times, then you spin up one thread per CPU, and "randomly" assign each task as it comes in to a thread.
When doing GUI style programming you have a lot of different tasks and each task is done in exactly one thread.
Hmm I would say the concepts are intertwined. Lots of GUI frameworks use async/await and the GUI thread is just another concurrency pattern that adds lock free thread exclusivity to async tasks that are pinned to a single thread.
Note that if you "just" write responses to queries without yielding execution, you don't need async, you just write Sync handlers to an async framework. (Hitting dB requests in a synchronous way is not good for your perf though, you better have a mostly read / well cached problem)
A particularly interesting use case for async Rust without threads is cooperative scheduling on microcontrollers[1]; this article also does a really good job of explaining some of the complications referenced in TFA.
It really is, but I still favour "unsexy" manual poll/select code with a lot of if/elseing if it means not having to deal with async.
I fully acknowledge that I'm an "old school" system dev who's coming from the C world and not the JS world, so I probably have a certain bias because of that, but I genuinely can't understand how anybody could look at the mess that's Rust's async and think that it was a good design for a language that already had the reputation of being very complicated to write.
I tried to get it, I really did, but my god what a massive mess that is. And it contaminates everything it touches, too. I really love Rust and I do most of my coding in it these days, but every time I encounter async-heavy Rust code my jaw clenches and my vision blurs.
At least my clunky select "runtime" code can be safely contained in a couple functions while the rest of the code remains blissfully unaware of the magic going on under the hood.
Dear people coming from the JS world: give system threads and channels a try. I swear that a lot of the time it's vastly simpler and more elegant. There are very, very few practical problems where async is clearly superior (although plenty where it's arguably superior).
> but I genuinely can't understand how anybody could look at the mess that's Rust's async and think that it was a good design for a language that already had the reputation of being very complicated to write.
Rust adopted the stackless coroutine model for async tasks based on its constraints, such as having a minimal runtime by default, not requiring heap allocations left and right, and being amenable to aggressive optimizations such as inlining. The function coloring problem ("contamination") is an unfortunate consequence. The Rust devs are currently working on an effects system to fix this. Missing features such as standard async traits, async functions in traits, and executor-agnosticism are also valid complaints. Considering Rust's strict backwards compatibility guarantee, some of these will take a long time.
I like to think of Rust's "async story" as a good analogue to Rust's "story" in general. The Rust devs work hard to deliver backwards compatible, efficient, performant features at the cost of programmer comfort (ballooning complexity, edge cases that don't compile, etc.) and compile time, mainly. Of course, they try to resolve the regressions too, but there's only so much that can be done after the fact. Those are just the tradeoffs the Rust language embodies, and at this point I don't expect anything more or less. I like Rust too, but there are many reasons others may not. The still-developing ecosystem is a prominent one.
I read comments like this and feel like I’m living in some weird parallel universe. The vast majority of Rust I write day in and day out for my job is in an async context. It has some rough edges, but it’s not particularly painful and is often pleasant enough. Certainly better than promises in JS. I have also used system threads, channels, etc., and indeed there are some places where we communicate between long running async tasks with channels, which is nice, and some very simple CLI apps and stuff where we just use system threads rather than pulling in tokio and all that.
Anyway, while I have some issues with async around futur composition and closures, I see people with the kind of super strong reaction here and just feel like I must not be seeing something. To me, it solves the job well, is comprehensible and relatively easy to work with, and remains performant at scale without too much fiddling.
Honestly, this is me too. The only thing I’d like to also see is OTP-like supervisors and Trio-like nurseries. They each have their use and they’re totally user land concerns.
> It really is, but I still favour "unsexy" manual poll/select code with a lot of if/elseing if it means not having to deal with async.
> I fully acknowledge that I'm an "old school" system dev who's coming from the C world and not the JS world, so I probably have a certain bias because of that, but I genuinely can't understand how anybody could look at the mess that's Rust's async and think that it was a good design for a language that already had the reputation of being very complicated to write.
I'm in the same "old school" system dev category as you, and I think that modern languages have gone off the deep end, and I complained about async specifically in a recent comment on HN: https://news.ycombinator.com/item?id=37342711
> At least my clunky select "runtime" code can be safely contained in a couple functions while the rest of the code remains blissfully unaware of the magic going on under the hood.
And we could have had that for async as well, if languages were designed by the in-the-trenches industry developer, and not the "I think Haskell and Ocaml is great readability" academic crowd.
With async in particular, the most common implementation is to color the functions by qualifying the specific function as async, which IMO is exactly the wrong way to do it.
The correct way would be for the caller to mark a specific call as async.
IOW, which of the following is clearer to the reader at the point where `foo` is called?
Option 1: color the function
async function foo () {
// ...
}
...
let promise = foo ();
let bar = await promise;
Option 2: schedule any function
function foo () {
// ...
}
let sched_id = schedule foo ();
...
let bar = await sched_id;
Option 1 results in compilation errors for code in the call-stack that isn't async, results in needing two different functions (a wrapper for sync execution), and means that async only works for that specific function. Option 2 is more like how humans think - schedule this for later execution, when I'm done with my current job I'll wait for you if you haven't finished.
Isn't mixing async and sync code like this a recipe for deadlocks?
What if your example code is holding onto a thread that foo() is waiting to use?
Said another way, explain how you solved the problems of just synchronously waiting for async. If that just worked then we wouldn't need to proliferate the async/await through the stack.
> Said another way, explain how you solved the problems of just synchronously waiting for async.
Why? It isn't solved for async functions, is it? Just because the async is propagated up the call-stack doesn't mean that the call can't deadlock, does it?
Deadlocks aren't solved for a purely synchronous callstack either - A grabbing a resource, then calling B which calls C which calls A ...
Deadlocks are potentially there whether or not you mix sync/async. All that colored functions will get you is the ability to ignore the deadlock because that entire call-stack is stuck.
> If that just worked then we wouldn't need to proliferate the async/await through the stack.
> and not the "I think Haskell and Ocaml is great readability" academic crowd.
Actually, Rust could still learn a lot from these languages. In Haskell, one declares the call site as async, rather than the function. OCaml 5 effect handlers would be an especially good fit for Rust and solve the "colouration" problem.
I think Rust’s async stuff is a little half baked now but I have hope that it will be improved as time goes on.
In the mean time it is a little annoying to use, but I don’t mind designing against it by default. I feel less architecturally constrained if more syntactically constrained.
I'm curious what things you consider to be half-baked about Rust async.
I've used Rust async extensively for years, and I consider it to be the cleanest and most well designed async system out of any language (and yes, I have used many languages besides Rust).
Async traits come to mind immediately, generally needing more capability to existentially quantify Future types without penalty. Async function types are a mess to write out. More control over heap allocations in async/await futures (we currently have to Box/Pin more often than necessary). Async drop. Better cancellation. Async iteration.
Actually, this "old school" approach is more readable even for folks who have never worked in the low-level C world. At-least everything is in front of your eyes and you can follow the logic. Unless code leveraging async is very well-structured, it requires too much brain-power to process and understand.
You cancel a sync IO op similar to how you cancel an async one: have another task (i.e OS thread in this case) issue the cancellation. Select semantically spawns a task per case/variant and does something similar under the hood if cancellation is implemented.
You can do that, but then the logic of your cancellable thread gets intermingled with the cancellation logic.
And since the cancellation logic runs on the cancellable thread, you can't really cancel a blocking operation. What you can do is to let it run to completion, check that it was canceled, and discard the value.
Not sure I follow; the cancellation logic is on both threads/tasks 1) the operation itself waiting for either the result or a cancel notification and 2) the cancellation thread sending that notification.
The cancellation thread is generally the one doing the `select` so it spawns the operation thread(s) and waits for (one of) their results (i.e. through a channel/event). The others which lose the race are sent the cancellation signal and optionally joined if they need to be (i.e. they use intrusive memory).
He didn't say queues though. CSP isn't processes streaming data to each other through buffered channels, it's one process synchronously passing one message to another. Whichever one gets the the communication point waits for the other.
(This is assuming you are already switching to communicating using channels or similar abstraction.)