[taberiand]

There's a lot of talk in general how Rust has a steep learning curve, mainly due to the borrow checker and lifetimes - but is this as hard as it gets?

Everything here seems logical and fairly self-explanatory, the only slightly alien thing is the lifetime annotation.

Are there advanced gotchas that can actually trip up an experienced coder? Are there some aspects of real work that Rust makes difficult, or impossible?

I mostly code in C# (which in my opinion is an excellent and productive ecosystem), but Rust seems to make a promise of providing a greater level of confidence in the code that is very alluring - I'd like to take the plunge, I'm just a bit worried about smashing into any hidden rocks at the deeper end

[loeg]

I found a few things a little difficult. First, the compiler is not actually as smart / accepting of valid borrow patterns as should abstractly be possible. Things like borrowing different subfields of a struct in different and nonconflicting ways to different subroutines. This got a lot better in recent memory with non-lexical lifetimes but I think is still a problem.

Two: splitting mut borrows. It’s reasonable to take a mut borrow of some resource and divide it into two independent borrows of exclusive components. Think non-overlapping subarrays, or independent struct members. The language does not really provide this for you, but it is safe. (There are some APIs like this but last time I checked they didn’t cover all usecases.)

Structs containing a borrow are just weird and annoying to work with. Lifetime parameterization everywhere is verbose and it’s difficult to determine where you’ve made a mistake. The syntax for describing lifetime relationships is non-obvious to me.

[estebank]

> There are some APIs like this but last time I checked they didn’t cover all usecases.

This refers to[1]

    <&mut [T]>::split_at_mut(usize) -> (&mut [T], &mut [T])`

used as

    let (first_42, rest) = mutable_slice.split_at_mut(42);

but you can see that the inner logic is just a bit of pointer twiddling[2]

        let len = self.len();
        let ptr = self.as_ptr();

        // SAFETY: Caller has to check that `0 <= mid <= self.len()`
        unsafe { (from_raw_parts(ptr, mid), from_raw_parts(ptr.add(mid), len - mid)) }

[1]: https://doc.rust-lang.org/std/primitive.slice.html#method.sp...

[2]: https://doc.rust-lang.org/src/core/slice/mod.rs.html#1641-16...

[loeg]

Yes, exactly. I was using this for zero-copy serialization into a buffer. At the time, I don’t believe that API had been stabilized yet.

[estebank]

That method has been around since 1.0 (as documented in the source code). Were you using Rust before 2015? If so, a lot has changed :)

[loeg]

No, post-1.0. I don't remember exactly why I had to roll my own with unsafe, but it was essentially the same thing.

[bobbylarrybobby]

For someone looking for an example of this, here's a SO post: https://stackoverflow.com/questions/74592882/extend-a-string...

Tl;dr Rust can't distinguish between a `Vec` whose length is mutable (which might need to reallocate) and one whose elements are mutable (which can provide mutable references to its elements but will never have to move them), so it prevents mutation in a case where it would be safe.

[zozbot234]

Isn't the latter just a mutable slice?

[sidlls]

Well, the blog examples are quite trivial.

Take a look at the standard hashmap implementation (https://doc.rust-lang.org/src/std/collections/hash/map.rs.ht...). Notice the lifetime annotations everywhere. This is a relatively basic data structure, but it has a ton of visual noise due to the constraints of the lifetime and borrow model, and programmers must know, understand and be able to reason about these constraints when using this data structure.

Now imagine a case where you need to operate on values with two (or more) different lifetimes (by definition, all must live at least as long as the one with smallest lifetime, but in practice they all can and will have different lifetimes). Now you have `'a` and `'b` (and perhaps more) everywhere *and* developers must now reason about that. (random example pulled from a popular networking crate: https://github.com/tokio-rs/tokio/blob/718d6ce8cac9f2e081c0a...)

It isn't trivial to pick up from scratch, especially if one comes from a much higher level language (C#, Java, python, etc.). This is true even for people who are very experienced writing high-quality, durable, and safe code in other languages (e.g., C++) without the help of a compiler, because the ownership and lifetime model are both different and compiler-enforced.

[saghm]

> Take a look at the standard hashmap implementation (https://doc.rust-lang.org/src/std/collections/hash/map.rs.ht...). Notice the lifetime annotations everywhere. This is a relatively basic data structure, but it has a ton of visual noise due to the constraints of the lifetime and borrow model, and programmers must know, understand and be able to reason about these constraints when using this data structure.

I'm honestly not sure what you're referring to with this example. The lines you linked to have no lifetime annotations, just generic type parameters (which I expect you'd see in most statically typed languages for a type like this), and from scanning down the file, the only places I see explicit lifetime annotations other than the anonymous lifetime (i.e. '_) are in documentation comments until the iterator implementation. I already think that the somewhat common argument that Rust makes implementing data structures isn't really that meaningful (because despite implementing hashmaps and linked list being common CS homework assignments, it really isn't common for most programmers to do at their jobs), but I'm not sure that you can even make that point with regard to lifetimes here. If anything, I think being able to do almost all of the hashmap implementation without needing to explicitly name a single lifetime makes the opposite point, which is that you can actually go surprisingly far without needing to use lifetimes at all and that even an intermediate level understanding of how to use them will cover a large portion of the cases where you absolutely can't avoid them.

Lifetimes can definitely be confusing, and sometimes the syntax for them is verbose enough that it can make it hard to understand what's going on, but I don't think they're quite as big a roadblock as they might sound from your comment. Even when you do have to use them, there's a dirty hack you can use if you're absolutely stuck (assuming you're writing safe Rust; unsafe is a whole other can of worms that I think you can make a good case about with regards to data structure implementation, but it's also even less likely that you'll need to use it than explicit liftetimes): the compiler only requires explicit lifetimes because brute forcing the potential lifetime combinations to see if any is valid is extremely inefficient, but you can always just try the different combinations yourself. Because the whole point of providing explicit lifetimes is that the borrow checker can easily verify whether the ones you provided are correct or not, you don't actually have to be able to tell yourself! By definition any lifetimes that the compiler doesn't give errors on are correct, so if you find something that works, you can move on (and maybe later ask online for someone to explain why it works if you're curious, or come back later yourself when you feel more confident in your understanding).

[puffoflogic]

> Now you have `'a` and `'b` (and perhaps more) everywhere

Only if you're a bad rust programmer. Lifetime variables can and should be a descriptive identifier, just like literally any other variable. Complaining that there's lots of 'a and 'b around is like complaining that your code has lots of x, y, z variables and you can't keep them all straight... It's not a language problem.

[cwzwarich]

To be fair to people using 'a/'b-type variables:

1) They are used pervasively in the standard library and its documentation.

2) Due to lifetime subtyping and implicit bounds, descriptive names may be inaccurate. For example, if I have an Ast that contains slices with a lifetime of the source code, I might decide to write Ast<'source>. However, what do I do if I then have a reference to such a thing? Do I add another lifetime parameter, e.g. &'ref Ast<'source>? If I am not going to be returning any of the 'source-lifetime slices, this additional parameter may be redundant. In such a case, do I write &'source Ast<'source> or &'ref Ast<'ref>? Both of those names are incorrect. Do I keep around an extra lifetime parameter to get the correct names (and be more future-proof), or do I just suck it up and use 'a?

[puffoflogic]

This is a non-problem, or at least a non-unique problem. It is the exact same thing as using variables as function parameters. Sometimes what the variable means in one place is different from another place, so... you use different names. This surprises no one except maybe first-time programmers. Sometimes a variable means nothing and we can give it a single-character name, but then we shouldn't have a bunch of them to get confused by.

[fbdab103]

Nearly every Rust tutorial/blog/reference I have come across uses single letter names.

[sidlls]

It's a "problem" in the sense that it's additional syntax with critically important meaning to the understanding of the code. The descriptiveness of the labels may help, but it won't eliminate the additional cognitive overhead required to mentally parse and reason about the code.

[howinteresting]

Lifetime variance often makes it so that more descriptive names are misleading.

[kapolos]

Imho, Rust is not hard as in "high complexity hard", but it is hard as in "must practice to get good at".

All the little rules and new concepts add up and in the beginning it can be frustrating to have the compiler complaining. On the plus side, it's like a motorcycle - it doesn't take long to drive it safely & efficiently.

Learning Rust is fun, I wholeheartedly suggest it even if you don't end up using it at work.

[pitaj]

I don't think motorcycles are a great analogy. They're way more dangerous for the driver than a car and you have to be constantly vigilant about your surroundings.

[vladvasiliu]

Yeah. If you start to get a bit too confident for your level, you'll just get yelled at by the compiler instead of ending up in the ER (if you're still breathing).

But I suppose the analogy is more along the lines of "it can be intimidating at first, but you can quickly figure out how to use it reasonably well".

[kapolos]

You are right, I should have found a better one. I was between "bicycle" and "motorcycle" and I felt that bicycle would imply that learning is a breeze, so I went with motorcycle.

[ok123456]

The ownership rules make it very difficult to write data structures.

Even the most trivial singly linked lists have caused innumerable blog articles to be penned on how exactly to do that in the most idiomatic Rusty way.

Personally, I don't see any problem with using unsafe Rust in these instances, because linear types are not appropriate model as you don't have a single source and sink.

[fleventynine]

> Even the most trivial singly linked lists....

Uh, a singly linked list is trivial to implement...

  struct LinkedListNode<T> {
    value: T,
    next: Option<Box<LinkedListNode<T>>>,
  }

Any data structure where chunks of heap memory are owned by a single pointer (red black trees, singly-linked lists, vectors, deques, circular queues, etc) can be represented easily in safe rust.

Implementing a double-linked list is where things get tricky, as the nodes don't have a singular owner.

[likeabbas]

Yep. It’s even why the standard library implementation of a doubly link list uses unsafe! Which, I don’t think it a problem. If you’re writing a data structure, it will be under heavy scrutiny, review, and testing so we should be okay with bypassing the compiler checks.

[codedokode]

As I understand, in low-level languages the concept of ownership exists anyway, no matter whether the language allows to declare it explicitly or not. Rust just makes you write it out explicitly.

[ok123456]

Yes ownership exists, but the semantics of Rust's automatic ownership system are different as they're enforced as a linear type.

[tmtvl]

The only concept of ownership that exists in low level languages is that the kernel owns memory and gives your program a chunk of it. Aside from that I think the closest equivalent concept is the system break, but that just determines where in memory your stack and heap are divided.

[mtlmtlmtlmtl]

Ownership doesn't exist in the C language per se, but it is still an important concept you have to reason about when manually managing memory. I think that's what GP meant.

E.g if you add some object to a generic hash table as a key by pointer, you better treat that pointer as being owned by the hash table and not mutate it, or interesting stuff will happen.

[tmtvl]

I mean sure, but we're not talking about C, we're talking about low-level languages, like IA32 assembly.

[mtlmtlmtlmtl]

No one uses "low level language" to mean only assembler anymore. Besides, you also have to reason about ownership in assembler. It's not something you can really get away from at any level of the stack.

[tcfhgj]

> Even the most trivial singly linked lists....

Well you exactly named the most difficult to handle data structures for Rust.

There are plenty that are easy

[ok123456]

A linked list is the simplest data structure there is.

Is a Red-Black tree simpler than a linked list to implement than a linked list in Rust? What about a Fibonacci heap?

[howinteresting]

The simplest data structures are pairs (A, B) and Option<T>.

[ok123456]

Wouldn't call those data structures. Those are data types.

[howinteresting]

They are structured data. The restriction of the term "data structures" to what's asked in interview puzzles is ridiculous IMHO.

Edit: and even among "interview puzzle data structures", stacks and queues have much simpler semantics than linked lists.

[mtlmtlmtlmtl]

I think using data structures as an example is sort of missing the forest for the trees. It's definitely harder to do, but you generally don't need to as there's an excellent standard library and wider ecosystem of libraries that do this for you.

And implementing data structures in any language is hard. It's just a harder thing to do than most things because there's always gonna be a lot of gnarly implementation details and edgecases.

[ok123456]

> implementing data structures in any language is hard.

Why make it harder.

> excellent standard library.

The standard library is intentionally bare-bones. They don't want to maintain libraries in std, and want you to use crates.

> wider ecosystem of libraries

The quality of these is quite poor in a lot of cases. People literally make blog posts into crates.

[antonvs]

> Why make it harder.

To achieve static memory safety.

If you don’t need or want that, you’re probably better off sticking to other languages. Although hopefully, not something like C or C++ which the US government now advises against using for security reasons.

[mtlmtlmtlmtl]

As someone coming from C I find the stdlib is fine. It has all the datastructures you need for the vast majority of code. It doesn't have very domain specific things like the kind of structures a text editor might use, for instance. That's fine.

As for the quality of crates, yes, this is the case in any language. Using a library includes the responsibility of making sure the quality is up to scratch. This is not unique to Rust.

[bitwize]

The simplest data structure there is (aside from scalars) is the array or vector.

Which you should be using anyway instead of linked lists so you don't blow up your cache.

[mplanchard]

I came to rust mostly from higher level languages, and I didn’t have as much of a hard time with lifetimes as I did with dynamic dispatch via trait objects. Lifetimes can certainly get hairy, but it’s easy to get around them when you’re learning by cloning a value, wrapping it in an Arc, or whatever. I have had multiple times where I wanted to use dynamic dispatch and discovered after a few days of work that my idea wouldn’t work for one reason or another. Mostly this came down to some limitations on what trait objects can and can’t do, but it has mostly ceased to be a problem now that I understand those limitations better.

[KingOfCoders]

You didn't have a hard time with the borrow checker because you used the Rust GC? Was that the argument?

Actually I left Rust and moved to Go although I love Rust much more because every struct ended up sprinkled with Arc.

[mplanchard]

No, there’s no argument. The statement was that lifetimes aren’t that hard, and that when you do get stuck there are escape hatches. The escape hatches are especially useful when you’re still learning.

FWIW I write Rust professionally now, and in our entire codebase we have maybe four or five data structures containing Arcs, specifically where objects are spawned as Futures on other threads.

[KingOfCoders]

That would be a very interesting metric, could you grep|wc them and the locs?

[mplanchard]

Sure!

Tokei says that we've got ~76k lines of Rust.

I used ripgrep to find occurrences of `Arc::` (to exclude type signatures and only find constructors) and then just gave it a quick manual look to exclude tests and benchmarks.

This leaves me with:

- one data structure that contains general application state and contextual information, which is created at application startup and shared across all tasks/threads

- one data structure for accumulating results that is shared across threads. This is an Arc<Mutex<u64, SomeEnum>>. The less common of the two enum variants is an Arc<BTreeMap>.

- one data structure used to pass a database connection in to a context where it'll be used to spawn async tasks

All told, we've only got four places in production paths where we construct a new Arc. One of those four is only called once in the life cycle of the application, while the others are called for any given invocation.

[KingOfCoders]

You're comment made my day, thanks for your work!

[pclmulqdq]

Cloning values and using Arc to wrap structs carry a huge performance costs. It's pretty easy to write correct Rust. It's a lot trickier to write fast Rust.

That's where the learning curve started to get very steep for me.

[fbdab103]

Is the performance difference meaningful? I write Python, so I suspect even poorly ported Rust will outperform artisanally crafted Python.

[pclmulqdq]

Yes. Most of the production use cases for Rust are aimed at C, C++, and Go users. In my past experience, even poorly-written Go tends to outperform poorly-written Rust for many tasks, not to mention C or C++.

You're not exactly the target market if you're writing python.

[fbdab103]

But that is kind of my point. There are plenty of reasons to use Rust that have nothing to do with performance. My brief toe-dip into the Rust world loves the static types, good dependency management, and single executable deployment. That it is faster than my standard language is just the cherry on top.

[mplanchard]

Yeah, Rust is a really fun language to write in my opinion. If your use-cases are tolerant of non-hyper-optimized Rust, there’s no reason to make it too hard on yourself while you’re learning. As you use it, I think you’ll naturally gravitate towards writing more optimized code, because the language guides you in that direction.

[pclmulqdq]

The unfortunate reality for many Rust fans is that a lot of people would prefer to use a less complicated language when performance doesn't matter.

That Rust program you spent a day golfing to make beautiful use of iterators? Your friend wrote it in go in 15 minutes.

They also get static types, good dependency management, and singe executable deployment. Cython + mypy basically gives you that with Python.

[mplanchard]

This conversation was about learning Rust, not about writing maximally optimized Rust. Clones and Arcs can make learning a lot easier, since they let you get stuff done without needing to figure out every obscure lifetime error.

For production contexts, we take more care to optimize for performance.

[pclmulqdq]

You and I have very different definitions of "maximally optimized." I think of eliminating all possible wasted CPU cycles when I hear those words. In comparison, being able to remove spurious copies and atomic accesses is table stakes for claiming that you know a systems programming language. Most use cases of C and C++ today are in that state.

[tmtvl]

Yeah, but Python is the slowest language ever. I think that a decent language like D or Common Lisp would outperform poorly written Rust, and they're easier to handle.

[estebank]

It really depends on what you're building whether that perf cost is problematic or not, and usually it isn't. If 90% of your code is clean Rust and the last 10% outside of the critical path is a straightforward clone or Arc, then I see no reason not to go that way.

[PoignardAzur]

> and using Arc to wrap structs carry a huge performance costs

Does it? It's like using shared_pointer, which some C++ codebases do everywhere for memory safety.

And some languages like Swift even do it by default.

[mplanchard]

Huge is maybe a strong term. The cost of a clone is hugely dependent on the size of the data being cloned. Avoiding clones of large data structures is important, but even that is unlikely to be a bottleneck outside of a hot path.

Arcs can be expensive, but once you’ve got the sense for lifetimes, they aren’t that hard to avoid.

[ninkendo]

> Mostly this came down to some limitations on what trait objects can and can’t do, but it has mostly ceased to be a problem now that I understand those limitations better.

Care to recount what some of those misunderstandings were? I’m casually interested in Rust but only really observe from afar, since most of my day job is Swift. Don’t get me wrong, Swift has some odd limitations around protocols (closest equivalent to traits) that may be similar, but I’m curious to see what some common pitfalls may be with Rust traits.

[mplanchard]

I’m sorry I don’t have specific examples because it’s been a while, but IIRC generally the issues came from trying to mix compile-time dynamism with runtime dynamism. Things like depending on methods that made traits non-object-safe and so on.

I think it can be quite confusing initially how traits are both the unit of compile-time generics AND can be used for dynamic dispatch at runtime, given that there are separate rules about what can be used in which context.

[mtlmtlmtlmtl]

I think Rust is a lot easier when you already have decent experience with C. In C you have to reason about ownership and lifetimes too, the only difference is it's all implicit. Knowing C also helps you a great deal in the cases where unsafe is either neccessary or just the best solution. Programmers who haven't touched C might be very hesitant to even consider using unsafe.

Other than that, it's been my experience that lifetimes come up very rarely or even never for a large range of programs. But someone coming from Java/C# where almost everything is a reference might struggle more than others because these languages lend themselves to constructing large object graphs that are annoying to do in Rust. You have to take a different approach, keeping object graphiness to a minimum where possible.

[insanitybit]

Unless you're doing something tricky, this is basically it. If you want to implement data structures like cyclic graphs or doubly linked lists things could be trickier. Sometimes people struggle with async because they dive right into that early and it can be a bit tricky if you use async + closures and don't know what you're doing.

[howinteresting]

The biggest thing for me was not so much the ownership rules themselves but how to design your code structure around them.

[estebank]

I think this exactly it: the rules on their own are simple but their implications are far reaching and hard to recognize ahead of time until you practice writing code.

[MuffinFlavored]

Having to clone a String every time you send it around is “confusing” if you come from a C char* pointer background.

You can dereference it to &str but I’m pretty sure that introduces weird “lifetime” &’a or whatever errors in most places (when you are first learning to write Rust that is)

same for structs, I typically just derive Clone. Kind of gross but for hobbyist projects where I want to “move fast” (coming from node.js so it’s hard to not treat Rust like a “scripting” language), it suffices.

[marcosdumay]

That is pretty much it; it is way harder to learn to fit withing those constraints than you are assuming; and it has some very unintuitive consequences that won't appear on 99% of your code, but are very hard to deal with on the other 1%.

[revskill]

Yes, i'm at awe when seeing .to_ref, to_owned,... and a bunch of alien methods, too.

[protortyp]

In my experience it just gets really dirty when you want to work on a single data structure (like a vector) in a threaded way.

[insanitybit]

Interesting, I feel like "share vector across threads" is an area that rust excels in. There's `split_at_mut` and `rayon`, and now with GATs you should be able to create a lending iterator too.

[protortyp]

Hm interesting, I have to look into GATs then. Anyhow it also got easier with scoped threads. But doing it the old way was always a pain point for me.

[insanitybit]

Scoped threads definitely help, yeah. Before that you'd want to use a scoped thread crate or rayon.

[oxff]

Borrow checking (lifetimes and references) are fairly easy for a beginner to Rust to grasp, I think it is basically a meme at this point that this stuff is hard. Entirely different thing if you are making some kind of async library though, that is when you will run into advanced gotchas.

Rather than a nice steep curve, its more like a threshold function.

[tialaramex]

Maybe. I found it very easy, but as you've seen some people really struggle. I think it matters a lot how you've mentally modelled what's going on, Rust fit how I assumed everything actually works anyway. I think C programs I wrote 20 years ago do many of the things I do in Rust today. I had lots of C background (enough to at least cosplay as an expert), lots of Java, some Go, and a whole bunch of other languages in my 30+ years of programming before I decided to learn Rust.

I learned C# after Rust, you say "mostly" C# but I guess it depends what else you've some experience in. If you have only worked in managed memory languages, Java, Javascript, Python, that sort of thing, then you should expect some struggle in Rust just because it isn't a managed language and so you are now responsible for making sure resource management happens.

In most of the unmanaged languages if you screw up your "punishment" is that the program has mysterious bugs, good luck - but in Rust the compiler will yell at you. So, the upside is obviously at least you know there's a problem, but the downside is that you're responsible for fixing the problem, and in managed languages you were not.

As to picking it up, well, it's Advent of Code starting December 1st, which is often taken as an excuse to either learn a new language or practice a preferred one you don't get to use, if you write software as a fun activity rather than only for $$$. You can assume some people will post Rust solutions you can read for inspiration, so that's an option. There are various free tutorials, I won't recommend any because that's not how I learned and our experience is likely very different.

One thing this article doesn't mention at all that might jump out at a C# programmer depending on other background, Rust has Move assignment semantics. This might work out to be something you didn't know you missed in C# but on the other hand you may find it a surprise, hard to say until you try it.

In Rust if I have a variable a_1, with a Thing, in it, and I make a new variable b_2 and I assign b_2 = a_1, that Thing from a_1 is moved to b_2. If I try to just use a_1 later, Rust won't let me, the Thing is gone, it was moved to b_2. I can put a different Thing in a_1 if I want, Rust is fine with that, but you don't get copies of the same Thing in different variables by default. If that makes sense, b_2 = a_1.clone() asks the Thing to provide a deep copy of itself.

Types can opt in to having Copy semantics like you'd see in most languages, but only if the type is literally just some bits, and in this case Rust will of course just copy the bits on assignment so that does what you'd expect. Basic types like the integers are Copy, whereas something like a String is not.

So this prevents that problem where you've got a Thing and you change some stuff about it and then put it in a Collection, and then you change more stuff, put it in the collection again, repeat until there are a dozen things in the collection - and then whoops, the Collection doesn't have 12 different Things in it, it has the same thing, Twelve times, with your last set of changes. In C# it's not difficult to do this (I wrote this bug last month without thinking) but in Rust you can't because once the Thing was moved into the Collection it's gone, so you can't change it, let alone add it to the Collection again.