[robocat]

This is not a standard memory leak, and would not have been avoided by using rust.

Edited and re-edited: I was too quick to presume commenter was just spouting the common “rust is a panacea” theme. Kernels are all about “unsafe” concurrent access and reentrant code, so rust is not a panacea. For this case of multi-threaded/multi-process access (presumably from ring-0 kernel code accessing shared kernel memory), using rust primitives to help prevent race conditions could make sense (smart pointers), because the code is unlikely to be performance sensitive and the feature is there to protect against a fairly extreme corner case (crazy ad hoc GC for cyclic graph of processes sending each other file descriptors). Reliable discussion on rust for kernel drivers here: https://security.googleblog.com/2021/04/rust-in-linux-kernel... Disclaimer: not a kernel nor rust dev. In past dabbled with embedded kernel debugging. I keep tweaking this edit, because it is complicated!

[tsimionescu]

By my understanding, Rust's ownership model would prevent concurrent access to the socket buffer garbage collector data structures without proper synchronization, which was the source of this bug.

This is in fact an example of a class of bug that Rust's compiler is uniquely able to protect from - other memory safe languages don't make guarantees about concurrent accesses at all - at least not Java, C#, Go, Python, Haskell, OCaml etc. Perhaps Ada does have something?

[elcritch]

Robocat is probably correct. Rust doesn't prevent race conditions, just data races. For example a Rust CVE due to a race condition: https://www.cybersecurity-help.cz/vdb/SB2022012101

This CVE appears to be due to a race condition despite using atomics, so likely this could've happened in Rust code. Really to implement this sort of GC I'd wager that unsafe rust would also be required unless an entirely different algorithm was used.

[robocat]

Also this is kernel code running in a kernel contex, so the code can’t just use std::sync::{Arc, Mutex}[1] because Mutex uses user-space pthread_mutex_lock[2]. The implementation is knarly™ multi-threaded kernel code so it would probably require unsafe custom code (rather than using kernel locking and concurrency-management mechanisms which were not used by the existing code for presumably valid reasons). The rust code could easily have had the same fault. Excepts from [3]:

  The VFS layer is a complicated beast; it must manage the complexities of the filesystem namespace in a way that provides the highest possible performance while maintaining security and correctness. Achieving that requires making use of almost all of the locking and concurrency-management mechanisms that the kernel offers, plus a couple more implemented internally
  the kernel may find itself with a set of in-flight Unix-domain sockets that are only referenced by unconsumed (and unconsumable) SCM_RIGHTS datagrams; at this point, it has a cycle of file structures holding the only references to each other.
  there is more complexity than has been described above and some gnarly locking issues involved in carrying out these operations. See Viro's message for the gory details.

rust is magic, but it doesn’t make us infallible.

[1] https://itsallaboutthebit.com/arc-mutex/

[2] https://stackoverflow.com/questions/5095781/how-pthread-mute...

[3] https://lwn.net/Articles/779472/

[tialaramex]

Today the Rust std::sync::Mutex type (on Linux) just uses a futex, not the unwieldy pthread_mutex_lock (which on Linux ultimately has a futex inside it anyway).

This why Rust's Mutex<[16; u8]> (a Mutex protecting an array of 16 bytes) is significantly smaller than C++ std::mutex (which doesn't protect anything itself). This was already true on Windows, and for a few months it has been true on Linux too.

But you're correct that Rust for Linux doesn't have std, and so doesn't have std::sync::Mutex. However it does have kernel::sync::Mutex which is reminiscent of the standard library Mutex, e.g. Mutex<T> is a thing, locking defaults to giving you a guard with access to the protected contents, and so on. But being the kernel it has unsafe methods that look way more dangerous than I'd be comfortable with. The Linux kernel already needs a mutex type (in C) and so kernel::sync::Mutex<T> builds on that.

[azakai]

That's my guess too. These GC data structures need to be accessed from multiple threads, if I understood TFA, which means they won't compile normally in Rust. That is exactly Rust doing its job and preventing bugs, but it means that the developer then needs to use unsafe (or find a workaround with runtime checks, at the cost of overhead).

[tsimionescu]

You're absolutely right, I had initially understood that revmsg with the MSG_PEEK flag was concurrently accessing GC data structures and presumably corrupting them.

Instead yes, this is essentially a logic bug in the presence of concurrency, and no programming language can help with those. It would have happened just as well with Software Transactional Memory or with Erlang message passing.

[azakai]

> By my understanding, Rust's ownership model would prevent concurrent access to the socket buffer garbage collector data structures without proper synchronization

Possibly. But the first question is whether the person writing this in Rust would have used unsafe. Without knowing more details here, it's hard for me to guess.

> other memory safe languages don't make guarantees about concurrent accesses at all - at least not Java

Well, Java does have synchronized methods. Those lock the entire class. You can imagine writing a "manager" class that encapsulates all the GC data structures here, and that would have made this perfectly safe in Java using existing language features.

Of course, that would have been slower - so, again, it is tempting to use unsafe approaches, even in a memory-safe language like Java, but then you do risk bugs like this.

But of course I do agree that Rust, even with some amount of unsafe, would be a far safer language than C!

[tsimionescu]

The difference though would still be that, if they don't use unsafe or proper synchronization in Rust, their code won't compile. In Java, their code will compile just the same whether they use `synchronized` or not.

Of course the Rust compiler can't force you to write correct synchronization, but it can at least prevent you from forgetting about synchronization entirely.

[mhh__]

D sort of does. We have a type qualifier for shared data that is picky about accesses but it's not completely there yet i.e. still requires some knowledge.

[eru]

Idiomatic Haskell would probably use eg Software Transactional Memory for concurrency; but that would be too high level for the kernel.

A Haskell-like language for kernel development would lean heavily on linear typing. Some prototypes have been made and used.