[thesuperbigfrog]

"The top three challenging areas of Rust for current Google developers were:

  * Macros
  * Ownership and borrowing
  * Async programming

"

Async programming is the area I would like to see the most improvement, especially in the standard library.

So much concurrent and parallel Rust code relies on third-party libraries because the standard library offers primitives that work but lack the "creature comforts" that developers prefer.

It would be really nice if the Rust standard library were to get structured concurrency similar to what Ada has:

https://en.wikibooks.org/wiki/Ada_Style_Guide/Concurrency

https://learn.adacore.com/courses/Ada_For_The_CPP_Java_Devel...

[no_wizard]

Color functions are just not the right road to go down. Something akin to Javas new green threads would be better, or Go style coroutines.

The path Rust is going means async becomes viral, and is something I dislike a lot about JavaScript[0] and other languages I’ve worked in[1].

I’d love to see Rust avoid this trap.

[0]: I work in TypeScript in actuality not sure which to use here. It’s certainly by far the language I’ve used the most in my career now.

[1]: I remember it infected Python too and it was a pain as well when I did Python development years ago.

[steveklabnik]

Do you have a way to accomplish this while still staying within the other various constraints that Rust's async is under? I don't believe it is reconcilable.

For those not aware of the history and looking for background, I laid it out here: https://www.infoq.com/presentations/rust-2019/ and here https://www.infoq.com/presentations/rust-async-await/

Those style systems are useful and have advantages, but they also have disadvantages. Not every tradeoff is a good call for every system, and that goes both ways in this scenario.

[no_wizard]

Make it an optional crate like `no-std` is, where you specify it in your cargo.toml

[littlestymaar]

I really wish people stopped using this concept of “function colors”, especially in the context of Rust because the `async fn`/“regular function” split is strictly equivalent to the `try_something()`/`something()` split (the first one being fallible and returning a `Result` in case of failure). `Result`s and `Option`s are coloring the stack in exactly the same way a `Future` does (and `async` is pure syntactic sugar on top of future).

So, someone may like exceptions and green threads more than `Result` and `async` (and this is a completely valid PoV, even though I personally like the explicitness better), but thinking `async` is somehow special is just a conceptual mistake.

Edit to give an little more substance to the parallelism:

If you want to call a fallible function inside an infallible one, you MUST handle the result. If you want to use `?` then your function MUST return a Result.

Symmetrically, if you want to call an async function from a non-async one, you MUST `spawn` the future. If you want to use `await` then your function MUST be async.

The only practical difference between async functions and functions returning a `Result` is that `Future` is a trait, not a struct like `Result` (and that means that your future may have a lifetime that's not visible in your function definition, which is an endless source of confusion for beginners).

[kaba0]

While you are technically right, one area where the two are unlike is that you want to be polymorph with regards to async/non-async, while it is less often the case for error handling. That exact same sleep implementation, or db query should be usable both as an async call, as well as a blocking one and it is the caller that wants to decide that. Which is not trivial to do on the calling side from a language perspective as it then has to recursively decide the same for all its subcalls.

The new experimental languages with effect types might be able to give us the best of both as they actually expose what you are talking about as an abstraction at the type level. We will see.

[ithkuil]

Yes. To make this more concrete: If your non-async function uses a parameter whose type is a dyn trait or a generic bounded by a trait, it can call the methods of that trait and thus potentially call different implementations of that trait.

Currently, none of those trait implementations can be async because that would change the function signature.

So the only option you have in a trait implementation that needs to call a library that happens to use async apis internallt, is to "block_on" the async. Unfortunately iirc blocking on an async like that is executor specific and your impl must pick a concrete async executor which may be different from the one used elsewhere in the program.

[littlestymaar]

> While you are technically right, one area where the two are unlike is that you want to be polymorph with regards to async/non-async, while it is less often the case for error handling.

The reason why we don't want to be polymorphic between fallible and “infallible” functions is that we put a clear social hierarchy between “properly handle error cases” and “panicking”. Using panics instead of `Result` make error handling much less cumbersome in Rust too, but we've clearly internalized that this isn't something you should do for real. Symmetrically, I'd argue that there's no much point to have both async and sync interfaces, if the user just want the quick and dirty approach, `spawn_blocking` is barely more typing effort than `unwrap`. The broad developer community (most of which having programmed well before Futures/Promises went mainstream) disagrees with me on that point, but that's a cultural thing.

Oh, and Result/Options aren't the sole “stack coloring” thing in Rust either: if you have an “owned” variable down the stack, then you need to either change your entire call stack to take the variable “by ownership” instead of “by reference”, or you can `Clone` it.

And you known what's even worse than this: `&mut`, because then you have no quick-and-dirty fallback (cloning and mutably borrowing the clone variable means you're now dealing with a reference that has a much shorter lifetime than the original one and it only works if your reference doesn't leave the current scope).

As a personal anecdote of someone that's been doing Rust full time for 6 years now, I've encountered the `Option`/`Result `/“owned”/`&mut` function coloring problem many times, and exactly zero times the “async/blocking” function coloring problem. Yet for some reason people are obsessed by an old rant about JavaScript's callback hell ¯\_(ツ)_/¯

[kaba0]

Let’s not make this discussion so rust-specific — panics are not used that way in rust for a reason. But exceptions (especially checked exceptions like in java) don’t have that problem, and are exact analogues to Result types. The point is, some way or another that parse function can fail and you may want to handle it. The caller can easily decide that from afar.

This is not true of async/blocking — there can be semantic differences between two, otherwise equivalent implementations, and it is a recursive problem as I mentioned — how should an async-block-async call chain work exactly? Java can decide it at runtime, while rust with its tradeoffs meaningfully can’t - but it is a net negative tradeoff in its case.

[littlestymaar]

> Let’s not make this discussion so rust-specific

But this is a discussion about Rust! And my entire point is that async/await is entirely consistent with Rust's overall design.

> But exceptions (especially checked exceptions like in java) don’t have that problem, and are exact analogues to Result types.

Unchecked exceptions don't have this problem (but checked exceptions do), and that's exactly my point. Async/await vs green-thread is exactly the same trade-off than Result vs exceptions: one is “simpler to use”, the other is “simpler to read”. After years of programming, I personally came to the conclusion that we spend more time reading code than writing it (and it's going to be even more true in the near future with LLMs) so I lean on the Result/await side of things, but I don't have fundamental objections against green thread and exceptions.

I do have a fundamental objection against the idea that “async” is somewhat special.

> This is not true of async/blocking — there can be semantic differences between two, otherwise equivalent implementations

Result vs exceptions have also a significant semantic difference: unwinding, and especially the fact that you can trigger unwinding at any point. This is a significant issue when you're dealing with pointers.

> how should an async-block-async call chain work exactly?

I don't really understand this question. An async function is just a regular function that returns a Future, and yes since there's no marker for blocking function you can definitely call one inside an async context, even though it's often a very bad idea from a perf PoV (well it depends, locks are mostly fine, but you need to use them with caution).

In fact, the `async` marker in functions doesn't bring much (again “async function are just regular functions that return a Future”), and it would make much more sense to have a `blocking` stack-contaminating marker on function that call a blocking syscall in order to avoid performance problems due to those, but we can't have nice things because something Path Dependence something…

[valzam]

The main difference to me is that Async/await tends to permeate your whole codebase. Once one part of the system is Async/await everything is. With Go I can write most of my code synchronously and maybe somewhere down the stack make 3 HTTP calls in parallel without having to change anything about the calling functions.

[littlestymaar]

> make 3 HTTP calls in parallel without having to change anything about the calling functions.

Exception that now you need to bubble up the error condition comming from these functions (if you function didn't have other error already).

And in fact, adding those call do change things about how the calling function is run (yields point are inserted and the function isn't being run sequentially anymore), it's just not visible in the code, exactly like exceptions vs explicit error return values.

[littlestymaar]

> Exception that

Except*, damn autocorrect.

[rascul]

> not a struct like `Result`

Minor nitpick, but it's an enum.

https://github.com/rust-lang/rust/blob/master/library/core/s...

[littlestymaar]

Indeed, thank you. (and since it's now been 2h since I posted this, my stupid mistake will live forever)

[shellac]

For many languages I absolutely agree: stackful coroutines are the way to go since the programmer experience is much smoother, with fewer hiccups like having different kinds of functions, or being unable to yield in loops. Lua, Go, and now Java got this right; python, javascript, and c# have to live with a bit of a mess.

But Rust is not a language which can dictate its execution environment. It needs to be able to exist in a C-ish world, and that's not something that supports yielding. It's a shame, but at least you can write kernel modules in Rust.

[thesuperbigfrog]

>> But Rust is not a language which can dictate its execution environment. It needs to be able to exist in a C-ish world, and that's not something that supports yielding. It's a shame, but at least you can write kernel modules in Rust.

Ada is also used in embedded and low-level environments where the execution environment can be limited. The way that Ada "gets around" such limitations is through language "annexes". Annexes are optional language extensions for specialized use cases:

https://www.cambridge.org/core/books/abs/programming-in-ada-...

http://www.ada-auth.org/standards/22rm/html/RM-1-1-2.html

Rust partially does this already with [no_std] (https://docs.rust-embedded.org/book/intro/no-std.html), so the concept is not too different.

[kaba0]

> Color functions are just not the right road to go down. Something akin to Javas new green threads would be better, or Go style coroutines

I agree with you on this in case of high-level languages. Rust is not that, and wouldn’t be half as interesting that way — but by going the system/low-level language route it does have to make certain design decisions that are not ideal. They can’t do what java’s loom does as it requires knowing every method implementation, which is fine with a fat runtime, but is not possible in case of Rust, with plenty FFI boundaries, etc.

[nu11ptr]

While I agree and would prefer green threads to async myself, I believe Rust chose async/await because it was shown the former could not be done as a "zero cost" abstraction and didn't interop with C well (somebody correct me if I'm wrong on that).

[shadowgovt]

At this time, I believe Rust's approach is complicated but that's because correctly using "bare" threads is complicated. Goroutines simplify the problem but introduce runtime performance overhead that may not be suitable for applications Rust is used for.

(Generally, I avoid this problem these days by avoiding threads in favor of other abstractions or multiple processes communicating over an RPC channel).

[gpderetta]

I vastly prefer stackful coroutines than stackless, but of all the languages, rust is probably the one that can justify that decision the most.

[hgomersall]

"Color" in this case just means signature. Of course you can't mix signatures - why would you want to? You might as well argue that Rust shouldn't have had any more than a single integer argument to every function. Async in rust essentially desugars to a return trait impl.

[zaphar]

That isn't quite true. An async signature forces it's caller to be async as well. This is because of it's semantics. Most signatures don't infect their callers signatures like that. async is qualitatively different in this regard.

[hgomersall]

No it doesn't. Your can call an async function in rust from anywhere, you just can't await on it outside of an async function.

[zaphar]

The async function also can't await outside of an async runtime. Which means I cant just call it and expect it to work. I need to wrap the call in an executor of some sort. If the async function doesn't do any awaiting itself then it doesn't even need to be an async function.

So in practice my statement stands.

[hgomersall]

But that's true of everything. You can't use a struct except through an "executor" of some sort. Once you've created it, it sits there and does nothing until you exercise it through its methods or other methods that take it. In that case, there's no ambiguity because the type is explicit. I guess the problem is that really `async fn` looks like a function, but isn't really a function, it's a future. The semantics of a future are quite different to the semantics of a function. That's only a problem if you think the semantics of `async fn` should closely reflect a function.

The thing is, once you grok async in rust, other things make sense, like being able to construct futures by implementing `Future` on a struct. You don't actually need `async fn` to do async rust. It's just syntactic sugar, just like await is.

[lcall]

I'm no expert on all the details, but I found you can use (at least with tokio) runtime.block_on() to call async (sqlx) code from non-async (my) code. Ctrl-F to see my other comment about block_on, here.

[mjhay]

> Async programming is the area I would like to see the most improvement, especially in the standard library.

> So much concurrent and parallel Rust code relies on third-party libraries because the standard library offers primitives that work but lack the "creature comforts" that developers prefer.

This seems to be an repeated antipattern with a lot of languages/ecosystems, resulting in fragmented and half-baked solutions. A fully-featured async standard library involves making a lot of opinion-based decisions, and not everyone will be happy. But it's better for 80% of people who probably don't care that much, and nothing stops the other 20% from implementing libraries for their use-cases.

[pie_flavor]

As news of Rust's decisions several years ago reaches people only now, it is fun to watch people make predictions about 'the future' that have not been borne out in reality one smidgen. Just using the library everyone else uses has also worked for 80% of people that don't care that much.

[jimbob45]

Is there any Rust outfit out there that doesn't discourage macro use? For that matter, is there any team with a language out there that encourages macro use when working as a team?

[giraffe_lady]

I've never seen a team that encouraged writing new macros to solve routine problems. But I've certainly been on teams that made heavy use of a few carefully deployed macros to solve recurring problems specific to that codebase.

I think elixir's sigils are probably the closest thing I've seen to "routine, encouraged macro use." Since almost every application will end up with a bit of template lite almost-dsl pseudo language for something or other. They're simpler than defining a grammar & writing a parser and more maintainable than regex.

[59nadir]

I think Elixir is a bad example here because it's one of the eco systems that, while they preach "Use a function if you can!" very loudly, use macros much more heavily than other eco systems, often in places where they don't have to. Phoenix, the (unfortunate) flagship library, abuses macros all over the place where even relative beginners can see that they didn't need to (see [0] for example). It's incredibly badly designed overall and these things have set the tone (especially since a lot of Elixir programmers are in reality just Phoenix programmers).

So, while macros are "discouraged" in Elixir, in practice they are very much encouraged by several prominent libraries. Picking on Phoenix is very easy because it's so blatantly bad in this regard (and others) but it's almost impossible to do useful things with Ecto if you go outside the macro bubble, etc., as well.

Example that shows how an eco system that definitely could have done stuff with macros (Clojure) has correctly decided that writing functions that take data is better than using macros:

Elixir and `Plug.Router`:

    defmodule MyRouter do
      use Plug.Router
    
      plug :match
      plug :dispatch
    
      get "/hello" do
        send_resp(conn, 200, "world")
      end
    
      forward "/users", to: UsersRouter
    
      match _ do
        send_resp(conn, 404, "oops")
      end
    end

Clojure and `reitit` (https://github.com/metosin/reitit):

    (def router
      (r/routes
        [["/hello" {:get (fn [r] {:status 200 :body "world"})}]
         ["/users" {:name :users
                    :router users-router}]
         ["*" {:get (fn [r] {:status 404 :body "oops"})}]]))

P.S. I've used Elixir since 2015, this is not an opinion I've developed at a glance.

[rockemsockem]

So far I've only written rust on solo projects, why is macro use often discouraged? Does that discouragement extend to little things like derive?

[VWWHFSfQ]

rust team needs to abandon their own execution runtime and just bless tokio and pull it into std. They're doing nobody any favors right now

[Animats]

I'm struggling to get tokio out of my executable. I'm not using it, but stuff keeps pulling it in. Async contamination is a huge problem. For highly concurrent code with threads running at different priority levels, if async gets in there it makes a mess.

[jrockway]

You know what they say about libraries; if you're not having problems, you're not using enough of them.

On a more serious note, "I want other people to write my code, but they're not following my standards" is rarely a sympathetic point of view.

[arijun]

Sorry if this is ignorant, but what’s the difference between async and concurrent? Is the problem that async schedules everything itself?

[Animats]

"Async" is optimized for the special case of a server that is I/O bound and spends most of its time waiting for network traffic. This covers most webcrap, so many people who do nothing else want that. It also works like JavaScript, a model with which many web programmers are comfortable.

It's a bad fit if you have enough compute work to keep all the CPUs busy. Then you're dealing with thread priorities, infinite overtaking and starvation, fairness, and related issues.

[flylikeabanana]

Isn't this the difference between concurrency and parallelism? Like you said, Tokio (I'm coming from effect systems in Scala, the current generation of which take heavy inspiration from Tokio) is good for informing your program when your code is blocked so it can perform some useful work elsewhere, which is a fundamentally different problem to parallelizing code. So if I'm understanding your complaint right it's that Tokio sneaks in when its concurrency features aren't particularly useful for parallelization?

[Animats]

Correct.

I have an unusual application, a metaverse client for big 3D worlds. It has to deal with a flood of data while maintaining a 60 FPS frame rate. It's essential that the rendering thread(s) not be delayed, even though other background threads are compute bound dealing with a flood of incoming 3D assets. This does not fit well with the async model.

This sort of thing comes up in games, real time control, and robotics, but is not something often seen in web-related software.

[carlmr]

Not OP, but I think the problem they are trying to explain is that if you create an async function it can only be called from other async functions, so it's quite an infectious concept.

If you create a library that uses async, you're forcing everybody that uses the library into async as well (with the same executor).

If somebody writes a library now that's generally useful but uses async, it forces others to use async or rewrite the library themselves.

On the one hand a lot of people put this down as whining about free code, which is somewhat true, but the infectious nature makes the whole ecosystem less useful if you want to build something non-async.

[estebank]

The Rust team doesn't have its own execution runtime. Tokio is certainly the closest thing to the "default".

[sanxiyn]

Given that Linux kernel is using async Rust which is implemented on top of kernel workqueue, and tokio won't be usable in kernel, it is a good thing tokio is not blessed. That's what enables both userland and kernel to use async Rust.

[jenadine]

There can be both a "blessed" executor, while staying optional for such cases.

For example, Rust std lib has blessed Mutex. Even though these can't be used in the kernel, it is still good to have them in the std lib for >90% of normal crates.

[steveklabnik]

That is never going to happen, for both good reasons and bad reasons.

[VWWHFSfQ]

Yeah I figured. Fragmentation by design.

[steveklabnik]

One person's "fragmentation" is another's "supporting multiple use cases is important, even if their requirements are divergent."