[steveklabnik]

A good post, thanks for sharing.

> I want to make one (mildly controversial) thing clear: rewriting a library for the sake of only using Rust is not good engineering.

Strong agree.

> A literal rewrite of a project to Rust is not interesting, it’s not useful, it just causes churn and splits ecosystems. Time would be better spent either working with existing solutions that already have the effort put in to make them correct or to come up with new green-field projects.

This... I'm not so sure about. It really depends on what your objective is. For example, if your goal is to learn, you're not going to cause churn, and you're not going to split ecosystems. Working on project you already know well is a good way to learn, because you can focus on the language, not the project.

This also isn't exactly a re-write, in my mind. I mean it is, and it isn't. This is because...

> The biggest problem when wrapping wlroots was defining the ownership model of the objects that wlroots exposes.

The pain here isn't a re-write, it's an integration with an existing system. That's a good reason to not use something different! I also think this is interesting because it demonstrates something that's often said in discussions about Rust, but mostly in the abstract, and that's that Rust's rules influence the design of your system. It will guide you away from designs where the ownership of components is unclear. To many people, this is a benefit, but it can often cause struggles when learning. And, it can often cause struggles in situations like this: where you can't really re-write some external component to fit in the rules.

That being said, there should be a way to do the ownership part that makes sense, but I don't know wlroots well enough to comment.

That being said

> Currently there is 11 THOUSAND lines of Rust in wlroots-rs. All of this code is just wrapper code, it doesn’t do anything but memory management.

is also super legit. Managing this kind of thing is a pain. I can certainly understand not wanting to do it.

[ChuckMcM]

I agree and was going to say the same. That there was a huge challenge in writing code to manage memory ownership, is not surprising because memory ownership is so fluid and adhoc in C and its derivatives. And when you are asked to make that ownership explicit, if the original designers hadn't been thinking about it, you get a lot of cases which they would not have considered.

I would like to see a compositor written in Rust, I think it would make for a robust window system. I also know that starting from the position of "this will be written in Rust" would force the issue of memory ownership to the fore and result in a different architecture from the start.

[ajross]

Not a wayland, wlroots or rust expert, but dollars to donuts says that it's not "memory" management[1] at issue, but resource allocation on the other side of the graphics driver. Vertex arrays, textures, framebuffers, shaders et. al. all need some kind of allocation strategy, can in the modern world often be shared between process contexts, and they really don't fit well into the metaphors of either C or Rust.

But where in C you can just do it anyway, Rust is likely to flip out over your wrapper object abstractions.

[1] I mean sure, technically it is memory management, but you know what I mean.

[pcwalton]

I have a very simple GPU abstraction in Pathfinder that has to manage lots of these objects, and it was easy and natural to get them to work. Sometimes I have to throw in a reference count here and there to make sure things are destroyed in the right order.

If the model doesn't fit perfectly into Rust's ownership system, I usually just do the checks dynamically and panic, or accept logic bugs (but not memory safety bugs), if things go wrong. Is that admitting failure? Perhaps. But it's also just choosing my battles. I'm trying to keep my code clean and ship, not prove it correct. Memory safety problems are pernicious enough that it's worth significant effort to rule them out, but almost everything else is a judgment call.

[klodolph]

This is also something that causes some pain when using C++ with RAII. Logically it makes some amount of sense that you have e.g. a C++ wrapper for a texture in OpenGL, and it knows to call glDeleteTextures in its destructor.

HOWEVER, the destructor can now only be called if the OpenGL context is active, and the texture may be deleted if the context is lost (which can happen). At some point anybody who's tried to wrap an OpenGL texture with a C++ class either decides to accept that RAII doesn't completely work here, or refactors it so that you manage something like handles to textures, which is a bit silly because you are really at that point managing handles to handles to textures.

[ska]

re: the handles to handles stuff ... I don't think that's so silly really, when you are talking about a resource that in some fundamental sense belongs to a different system (in this case OpenGL). Sure, that system gave you a handle, but if the semantics of managing that handle aren't 1:1 matched with your languages model of things, another level of indirection is a clean way to handle it.

[anyfoo]

And just think about what most file and I/O wrappers, that are part of the standard library in most higher level languages, e.g. file objects in python, streams in C++ etc., naturally do: They also just wrap (for example) POSIX file descriptors, which are handles, with their own handles, i.e. the file object/Stream/whatever.

[klodolph]

Maybe I didn't explain what's happening in OpenGL properly, because that comparison doesn't work at all.

With e.g. std::ofstream, you are wrapping a handle, such as a POSIX file descriptor. So that part of your description is accurate.

However, this is different from what happens with OpenGL. The problem is that if you wrap an OpenGL texture (which is a handle), you end up with some problems because you can only free it with the correct context active, and it might become free for other reasons besides the wrapper being destroyed. So you are no longer wrapping a handle (like std::ofstream), but you are actually creating a new handle-to-handle, and wrapping that, and explicitly managing the lifetime OpenGL resources with a separate object somewhere else.

Of course, you could just wrap an OpenGL texture, explicitly decide not to handle context invalidation, and then be careful to ensure that your objects get destroyed with the OpenGL context active. You lose some flexibility and you have to babysit RAII to be sure it "does the right thing".

So what I'm saying here is that OpenGL textures are not like file handles.

[ska]

Right, it's the same basic problem, and a good pattern to solve it.

[sly010]

The problematic pattern I see here is whenever you manage remote state, your local guarantees about how that state changes just don't work.

You would love to have the abstraction of a standalone texture and be able to pass it around as a value and change it's ownership, but you never really owned it. The only thing you ever owned is the context (or the connection in the case of the wayland protocol). The texture you borrowed. Freeing your texture is equivalent to returning it to the context you borrowed it from. Of course not having an explicit context in the OpenGL API makes this much harder :)

[jandrewrogers]

Yes, a similar issue exists in C++ too when managing some types of resources in database kernels. Some types of resource management are a poor fit for idiomatic ("safe") resource management in C++. In that language, you always have the option of custom, non-idiomatic resource management -- everything you might want to do is always possible in principle. You avoid it to the extent practical but there are a few times when this kind of resource management is unambiguously the correct software design and engineering choice. It is a bit like "goto". Rarely used for good reason but there are times when any other implementation will be decidedly much worse.

I've hashed this out extensively when looking at implementing a database kernel in Rust (I’ve had Rust experts among my C++ programmers). Technically it is possible to implement these things in a safe way in both C++ and Rust. However, it comes at a cost of an unreasonably complex and inefficient software architecture in either language, so you would not want to actually do it that way in a sane code base.

[codedokode]

Wayland uses "objects" that can appear and disappear and graphic driver is unrelated here. For example, wayland has a "registry" and it sends notifications to clients when new objects appear there or disappear from it. I assume this unpredictability could become the problem for the author.

[timidger]

Author here.

>> A literal rewrite of a project to Rust is not interesting, it’s not useful, it just causes churn and splits ecosystems. Time would be better spent either working with existing solutions that already have the effort put in to make them correct or to come up with new green-field projects.

> This... I'm not so sure about. It really depends on what your objective is. For example, if your goal is to learn, you're not going to cause churn, and you're not going to split ecosystems. Working on project you already know well is a good way to learn, because you can focus on the language, not the project.

I was a little extreme with this comment, and I knew there were a bunch of footnotes like the ones you mentioned (pedagogically reinventing the wheel is great for learning!) I explicitly didn't mention those so I didn't water down my point.

Ultimately people can do what they want, and it won't really bother me. My comment was more of a critique on what is and isn't worth other's time. That is for them to decide, at the end of the day, it's just my opinion that I think they should question if all the effort they are going through is worth it. If they think it is, I wish them luck.

> The pain here isn't a re-write, it's an integration with an existing system.

I was conflating two ideas here that in retrospect I should have been clearer about.

wlroots-rs was definitely _not_ a RiiR, it was bindings. However as I came to butt against these problems it became obvious why other projects (like rlua and wayland-rs) are RiiR: because writing bindings is so difficult it is easier to start from scratch. At that point you are now rewriting a library for the sake of using Rust, which seems like a problem to me.

I don't know what Rust can do to make it easier to write these bindings. I think it's very important, but not something that has been focused on in the community because most of this work is very niche and there are other problems that are probably more interesting (async, web assembly, etc.)

[floatboth]

btw, wayland-rs is both RiiR and bindings currently (native_lib cargo feature), and I've used it with libweston for custom protocols!

My project https://github.com/myfreeweb/weston-rs was also abandoned though. For a much simpler reason — I realized that I didn't need to reinvent the wheel — https://wayfire.org is everything I could possibly want from a compositor :)

[sambe]

I'm surprised to see you agree strongly. A Rust re-write for the political/ideological reason of "only using Rust" seems not good engineering by definition - it is not an engineering decision.

However, most of the time that someone would rewrite for the sake of using Rust, it would naturally be to get the advertised benefits of the language. If you believe you will have safer, more manageable code in pure Rust, you are making a valid choice.

The fact that efforts will be split is unfortunate, and should factor into such decisions. But that doesn't seem a strong argument on its own - if you believe such a change is for the better then there's no way to avoid breaking some eggs.

[steveklabnik]

I think we're differing on semantics.

> it would naturally be to get the advertised benefits of the language

I think "naturally" is doing a lot of work here; this is not "for the sake of using Rust", it's to gain the benefits of the language.

[sambe]

I agree, but naturally I think that’s the natural interpretation :)

Speculating: I think many Rust rewrite projects are criticised under “for the sake of it” when in fact there is a clear decision that Rust is just a better choice in 2019 (i.e. the decision was not political, but the criticism assumed it was). Many of those will nonetheless peter out because forks are hard. Such a result is only a partial judgement on the original decision.

[steveklabnik]

> I think many Rust rewrite projects are criticised under “for the sake of it” when in fact there is a clear decision that Rust is just a better choice in 2019

Yes, I think that's true as well.

[GreaterFool]

I worked on projects that required complicated FFI bindings to C/C++ libraries. Creating, testing and maintaining such bindings always ends up a huge undertaking. Now I'd think twice. I imagine something like Wayland will have hundreds of functions and if one doesn't want to write C in Rust (which isn't that bad actually) then wrapping up such API is a Herculean task!

[fxfan]

I'm surprised nobody mentioned this yet- a rust rewrite means you are more likely to invite contributions from a larger community.

[codesections]

> rust rewrite means you are more likely to invite contributions from a larger community

I'd be really interested to know why you think that's the case. Somewhere around 3% of programmers know Rust, while over 20% know C. https://insights.stackoverflow.com/survey/2019#technology

Naively, I would have thought that a C project would invite contributions from a much larger community. Sure (as I've personally experienced!) the Rust community is friendly and helpful, but we're still small, so I'm curious why you think a rewrite would get more contributions.

[sgeisler]

Not parent, but I'd think twice before contributing to a C code base I'm not completely familiar with in fear of causing some bug/CVE. Rust projects are much nicer for drive-by contributions to projects one uses but isn't generally interested in working on. If you are not working on a security critical part of a system you can be pretty sure not to cause severe bugs like RCEs.

Also for pure rust projects the development environment is easier to set up: cargo+rustc do everything vs. autotools+make+installing system wide libraries+ccache+gcc (just what I've seen in the wild).

[ncmncm]

It is nice to think that 3% of programmers know Rust, but we all know that number is at least two orders of magnitude too large. It would be impossible to defend a claim that 3% of programmers know that Rust is a language.

[fxfan]

That's not how OSS contributions work. C# is wildly popular in enterprise, and most of New york runs on F#. How often do you see .NET repos on Github?

[codesections]

> How often do you see .NET repos on Github?

Well, ok, if we're going by GitHub repos, C has 117k repos with at least one star and Rust has 11k. That doesn't seem to be that different than then number of programmers as a whole.

(FWIW, there are 104k starred C# repos and 2k F# repos, which doesn't seem all that out-of-line with the popularity of those languages as a whole.)