[dxhdr]

> In fact the pattern described in the article is a common pattern in Rust and I make use of it all the time; the library for making use of it is `slotmap`.

Slotmap uses unsafe everywhere, it's a memory usage pattern not supported by the borrow checker. It's basically hand-implementing use-after-free and double-free checks, which is what the borrow checker is supposed to do. Is that really a common pattern in Rust?

[dralley]

> Slotmap uses unsafe everywhere, it's a memory usage pattern not supported by the borrow checker. Is disabling the borrow checker really a common pattern in Rust?

Wrapping "unsafe" code in a safe interface is a common pattern in Rust, yes. There is absolutely nothing wrong with using "unsafe" so long as you are diligent about checking invariants, and keep it contained as much as possible. Obviously the standard library uses some "unsafe" as well, for instance.

"unsafe" just means "safe but the compiler cannot verify it".

Unsafe does not disable the borrow checker, though. All of the restrictions of safe Rust still apply. All "unsafe" does is unlock the ability to use raw pointers and a few other constructs.

https://doc.rust-lang.org/book/ch19-01-unsafe-rust.html#unsa...

[tialaramex]

> Obviously the standard library uses some "unsafe" as well, for instance.

Most beautifully, MaybeUninit<T>::assume_init() -> T

This unsafe Rust method says "I promise that I actually did initialize this MaybeUninit<T>, so give me the T".

In terms of the resulting program the machine is not going to do any work whatsoever, a MaybeUninit<T> and a T are the same size, they're in the same place, your CPU doesn't care that this is a T not a MaybeUninit<T> now.

But from a type safety point of view, there's all the difference in the world.

Even though it won't result in emitting any actual CPU instructions, MaybeUninit::assume_init has to be unsafe. Most of the rest of that API surface is not. Because that API call, the one which emitted no CPU instructions, is where you took responsibility for type correctness. If you were wrong, if you haven't initialized T properly, everything may be about to go spectacularly wrong and there's no-one else to blame but you.

[dcow]

Exactly. People miss this all the time when they write off Rust for "needing unsafe to do real programming" or whatever uninformed criticism they're parroting (they've clearly never actually done this "real programming" in Rust). The whole point is to reduce the opportunity for unforced errors by marginalizing the cognitive load required for the programmer to ensure the program is correct. And a program with a few unsafe blocks to `assume_init` some memory that e.g. a driver initialized for you is still infinitely better in that regard than a program that's littered with `void*` everywhere.

[mike_hock]

> than a program that's littered with `void*` everywhere

Strawman argument. A properly written C++ program isn't littered with `void*` everywhere in the same way that a properly written Rust program isn't littered with `unsafe` everywhere. You build safe abstractions around the ugly low-level pointer handling, you just don't have a keyword for a clear delineation.

> People miss this all the time when they write off Rust for "needing unsafe to do real programming" or whatever uninformed criticism they're parroting

Hard-core Rust proponents also seem to miss this all the time. Because "you basically write the same unsafe code that you would write in C++ but you now have a keyword to mark it" just doesn't imply the same urgency for adopting the language than "you only need unsafe to implement a few primitives in the standard library" does, which always seems to be tacitly implied until called out, and then the critics are "misinformed."

[tialaramex]

Firstly the delineation clarity is much more valuable than you seem to appreciate. A day one beginner in Rust can see that this stuff is roped off - so they know if they should call a grown-up - and everything which isn't roped off is safe for them. This also benefits an experienced developer when you're not at your best. Lets not write unsafe Rust today, we can do that when the air conditioning works, the coffee machine is fixed and there aren't contractors using power tools in the office.

I also think you very seriously underestimate how much equivalently unsafe C++ you write, and overestimate how much actual unsafe Rust is needed. Philosophically WG21 (the C++ committee) didn't like safe abstractions, so it doesn't provide them. To the point where the C++ slice type std::span is exactly like the safety proposal where it was originally suggested, except with all the safety explicitly ripped out. "We like this safety feature, except for the safety, get rid of that". I am not even kidding.

Most Rust programmers don't need to write any unsafe Rust. They can rely on Rust's promises, about aliasing, races, memory safety, performance characteristics, and they have no responsibility for delivering those promises, it's all done for them so long as they write safe Rust.

The other crucial element is culture. Culturally Rust wants safe abstractions, that applies to the standard library of course, but it also applies to third party code, you can expect other Rust programmers to think your library is crap if it has a method which is actually not safe to call without certain pre-conditions but isn't labelled "unsafe" -- because that's exactly what "unsafe" is for so you're not fulfilling your social contract.

[pkolaczk]

> You build safe abstractions around the ugly low-level pointer handling, you just don't have a keyword for a clear delineation.

The main difference is they are not really safe. It is trivial to accidentally invoke UB with incorrect use of "safe" abstractions in C++ like built-in containers or smart pointers. Keep a reference to a vector element, add a new item to the vector and it will sometimes blow up ;)

[mike_hock]

I disagree that it is "trivial," at least in the example you stated. This take-reference-then-mutate is exactly the kind of usage that the borrow checker prevents. You have to avoid it systematically in both languages.

The built-in containers are also not the best examples of "safe" abstractions. You can build safer abstractions, and you can employ safer usage patterns of built-in vectors, at non-zero but marginal costs.

The honest view on C++ is that there is no such thing as "safe" in absolute terms, but you have a lot of tools to mitigate the unsafe nature of the core language.

The honest view on Rust is that the idea of categorically excluding memory safety errors didn't quite pan out, but we're nonetheless left with an improvement over C++.

[mr_00ff00]

If unsafe means “safe but the compiler cannot verify” then I guess just consider .cpp to mean “safe but the compiler cannot verify” and we have suddenly made C++ memory safe

[retrac]

There's a related idea in Haskell, usually considered a memory safe language. You can write a program in Haskell that directly mutates memory, or does IO operations, freely, anywhere in the code. This violates functional purity and the compiler cannot offer its usual promises; your program may very well segfault from a bug in such code. But sometimes you just have to, perhaps to implement an algorithm efficiently.

Still, it is discouraged; both culturally in the language community, and discouraged through the subtle prodding of the language itself (such as everything being typed "IO", or the slightly ominous "unsafe" in the "unsafePerformIO".) Very often, the amount of code that must truly live in IO can be reduced to a few dozen lines, if that. That code is crucial to get right -- it's where the actual sequence of computation and external effects are handled. Such isolation allows the rest of the code to not have to worry about those matters.

[jjnoakes]

Sure, and if a typical Rust program that I write has no unsafe in it directly, and 5% of its dependencies' code have unsafe in them, that's also the same as writing a program in the "not c++" language directly, and using "not c++" dependencies for all but 5% of the dependency code.

Seems like a silly analogy to me, though.

[mr_00ff00]

Right but it’s that 5% the origin comment is talking about. The times when rust has to use unsafe for the type of program.

[jjnoakes]

Unsafe Rust is safer than C++, and even if it wasn't, 5% unsafe in Rust programs (in well-marked locations) is vastly superior to 100% unsafe in C++ programs.

Any analogy that equates the two is silly.

[anonymoushn]

unsafe rust is less safe than C++ because of the provenance and aliasing semantics that unsafe rust must adhere to to avoid UB, which are generally tricker than those of C++

[umanwizard]

The core value proposition of rust is that it’s memory safe by default, and it’s possible to limit the set of code that needs to be manually checked for UB. This isn’t the case for C++, as any code anywhere can invoke undefined behavior.

[pjmlp]

True, as long static analysers aren't part of the build, at which time specific constructs can be made to break the CI/CD build, forcing everyone to play by the rules if they want the PR to go through.

It isn't perfect, but does improve a lot the security baseline.

[ammar2]

Sure but you're missing the

> so long as you are diligent about checking invariants

part. Could you go through and check all the parts of a huge C++ codebase to make sure invariants are held as opposed to a few hundred lines of unsafe Rust code?

[mr_00ff00]

Sure, but I think the point here is the degree.

Presumably if it takes a lot of unsafe rust lines to build something, it won’t matter if it’s 30% safe or whatever.

I just see the point of “unsafe is fine” a lot when the whole point of rust is that memory safety issues are never worth the cost.

[ammar2]

Right, I guess the question is what will that proportion be when Rust is used for things like operating systems and web browsers. 30% would be untenable but a few hundred/thousand lines of unsafe code is fairly easy to put under a microscope.

For some current day research into this, there is the paper "How Do Programmers Use Unsafe Rust?"[1] which I'll drop a quote from here:

> The majority of crates (76.4%) contain no unsafe features at all. Even in most crates that do contain unsafe blocks or functions, only a small fraction of the code is unsafe: for 92.3% of all crates, the unsafe statement ratio is at most 10%, i.e., up to 10% of the codebase consists of unsafe blocks and unsafe functions

That paper is definitely worth reading and goes into why programmers use unsafe. e.g 5% of the crates at that time were using it to perform FFI.

In writing "RUDRA: Finding Memory Safety Bugs in Rust at the Ecosystem Scale" [2], I recreated this data and year-by-year the % of crates using unsafe is going down. And for what it's worth, crates are probably a bad data-set for this. crates tend to be libraries which are exactly where we would expect to find unsafe code encapsulated to be used safely. There's also plenty of experimental and hobby crates. A large dataset of actual binaries would be way more interesting to look at.

[1] https://dl.acm.org/doi/10.1145/3428204

[2] https://taesoo.kim/pubs/2021/bae:rudra.pdf

[Gwypaas]

Or Rust in Android, in this deep dice gaining two places of unsafe code which found a bug in the existing implementation due to the vetting triggered by being the only two places.

https://security.googleblog.com/2022/12/memory-safe-language...

[mr_00ff00]

Ahh that is quite interesting, I’ll check those links out

[CJefferson]

Looking at a couple of programs I work on:

9,500 lines of code, 8 are unsafe.

7,000 lines of code, 22 are unsafe.

14,000 lines of code, 140 are unsafe.

As we follow the standard rust rule that "safe code should not be able to use unsafe code to do unsafe things", those unsafe bits of code have been very carefully checked, to the best of our abilities, to ensure they don't create memory safety issues. It is a lot easier to triple-check 170 lines of code than 30,000 lines.

[Animats]

Sharpview, my metaverse viewer: 36,000 lines, 0 are unsafe.

I use some published crates that have unsafe code, but my own programs start with

    #![forbid(unsafe_code)]

This is 60FPS 3D high-detail graphics stuff, where performance matters.

[cyber_kinetist]

A counterpoint that makes this argument a bit weaker: Rust’s “unsafe” marker doesn’t pollute only its scope and actually pollutes the whole module; You need to make sure that the invariants in unsafe code are met even in safe code. (An explanation of this in the Rustonomicon: https://doc.rust-lang.org/nomicon/working-with-unsafe.html)

So there’s quite a lot more code to actually check then what some of the Rust proponents are saying. One can say that C++ is still worse in this regard (theoretically you need to check 100% of your code to be safe in C++). But for some minority of developers who frequently needs to delve into unsafe code, the advantages of Rust might seem a bit more disappointing (“the compiler doesn’t really do that much for the more important stuff…”)

[kaba0]

> whole point of rust is that memory safety issues are never worth the cost

I don’t think that it would be the point of rust — otherwise why not write Java, or a litany of GCd languages instead?

Rust is a low-level/systems programming language where you have more control over the program’s execution (e.g. no fat runtime), which is a necessity in some rare, niche, but important use cases.

[dralley]

It almost never takes a lot of unsafe to build something. Even the redox OS kernel is only like 10% unsafe.

[Ygg2]

It's not what unsafe means. Unsafe means this might cause UB for some invocations (accessing raw pointers, calling into another language, etc.). Safe means it will not cause UB for any invocations (it may panic or abort).

[dxhdr]

It's essentially a "user-space" memory allocator with it's own use-after-free and double-free checks, apparently because the language implementation isn't adequate. If anything it just reinforces the articles point that "borrow checking is incompatible with some useful patterns and optimizations."

[junon]

Eh? This is a wild take. How do you draw the conclusion the default implementation is inadequate?

[dymk]

Because something like slotmap has to use `unsafe` to get around the inadequacies of the borrow checker...

[orlp]

Author of slotmap here.

There is absolutely no need for unsafe in slotmap. I chose to use unsafe (wrapped in a safe API) to reduce memory usage using intrusive linked freelists. If done using safe Rust this would involve `enum`s that would take up extra space.

[junon]

Slotmap is one of my favorite crates, by the way. Thank you for putting it out there!

[pjmlp]

Thus not adequate for performance requirements.

[burntsushi]

A downside for sure, but one that, at least in this specific example, has limited downsides. If you can button it up into a safe abstraction that you can share with others, then I don't really see what the huge problem is. The fact that you might need to write `unsafe` inside of a well optimized data structure isn't a weakness of Rust, it's the entire point: you use it to encapsulate an unsafe core within a safe interface. The standard library is full of these things.

Now if you're trying to do something that you can't button up into a safe abstraction for others to use, then that's a different story.

[runeks]

> "unsafe" just means "safe but the compiler cannot verify it".

"unsafe" means "safe"?

I would say "unsafe" means "only safe if used in a manner that cannot be checked by the compiler".

[ChrisSD]

There are two things here. The `unsafe` in an `unsafe { ... }` block is referring to the contents of the block. From the outside it is indeed safe to use as if it were safe code. No special requirements necessary. So, yes, from a certain point of view `safe` would have been a better name (albeit confusing in a different way).

An `unsafe fn` however does need to be used correctly (and should document those requirements). However, these can only be called within `unsafe` blocks, so see above.

[cyber_kinetist]

Not entirely correct, Rust’s “unsafe” marker doesn’t pollute only its scope, it actually pollutes the whole module; You need to make sure that the invariants in unsafe code are met even in safe code. (An explanation of this in the Rustonomicon: https://doc.rust-lang.org/nomicon/working-with-unsafe.html)

[orlp]

> Slotmap uses unsafe everywhere, it's a memory usage pattern not supported by the borrow checker.

Author of slotmap here. This is patently false.

Yes, the slotmap crate uses a lot of unsafe to squeeze out maximum performance. But it is not 'a memory usage pattern not supported by the borrow checker'. You can absolutely write a crate with an API identical to slotmap without using unsafe.

[dxhdr]

> But it is not 'a memory usage pattern not supported by the borrow checker'. You can absolutely write a crate with an API identical to slotmap without using unsafe.

I think that might actually be worse though, performance aside. You're performing memory / object lifetime management but the Rust borrow checker still would have no idea what's going on because now you've tricked it by using indices or an opaque handle instead of references. The program may compile just fine but could have use-after-free bugs.

At least with unsafe there's an explicit acknowledgement that the borrow checker is turned off.

[orlp]

Yes, using slotmap you can get "use after free"-style bugs that you would not encounter if you strictly stayed with the borrow checker. So if the borrow checker fits your purpose, by all means, go ahead.

But the borrow checker can't represent circular/self-referential structures you see very often in graphs. Nor is it convenient in some cases as it has a strict separation between references that can mutate, and those that can't, which doesn't fit all problems either because the mutable references are by necessity unique.

Note that a "use after free" in slotmap results in a None value, or a panic (exception for the C++ people), depending on which API you use. In other words, it is detected and you can handle it. It does not trigger undefined behavior, you don't get ABA-style spurious references, there are no security issues. It is not the same as the issues pointers have at all.

[chlorion]

I have implemented my own slotmap crate for a lisp interpreter that uses no unsafe code and provides exactly the same features as the "standard" slotmap crate.

There is nothing inherent to the slotmap that requires unsafe code! It's only used for optimizations purposes.

Mine works in a similar way to the "standard" slotmap. It's a vec of slots, slot is an enum that can be occupied or vacant, the occupied variant is a two tuple containing the value and generation, vacant holds just a generation. Inserting into the slotmap simply switches the variant of the slot from vacant to occupied, and popping does the reverse. If there is no currently vacant slots, we just use the underlying push method on the vec of slots which will handle resizing for us! I also store a stack of indexes to vacant slots to make insertion fast.

When you insert into the slotmap, it provides a opaque key, but the data inside is an index and a generation. When you attempt to retrieve a value with a key, the slotmap checks if the slot is occupied and if the generation matches, and if so returns the value, otherwise returns none.

There is also a indirect slotmap, that adds an extra layer of indirection, so rather than the key being an index directly into the underlying vec of slots, its an index into a vec of indexes, this allows moving the slots around without invaliding currently living keys.

The indirect slotmap has the advantage of faster iteration, since it doesn't have to skip over empty "holes" of vacant slots in the vec of slots. The tradeoff is that insertion is slightly slower!

Anyways, no unsafe is required to implement a performant slotmap data structure! I have not uploaded my slotmap to crates.io because I didn't think anyone would find it useful, but maybe I should reconsider this!