[cyphar]

Rust is much better for this (though I still feel some of the fork/exec interface has similar warts to Go). However, you're wrong in saying that it's "roughly as messy/clumsy" as C.

Let me tell you how runc works. runc is written in Go, and we take an OCI configuration file. Because we can't just fork and set up all of the namespaces in Go, we have a C function called nsexec which is specified as __attribute__((constructor)). This ensures that our code will execute before the Go runtime boots. The parent process writes (using netlink as the wire protocol) to a pipe that the child has open and is parsed in C. Then, the child will have to do a series of forks, unshare, setns, {open,read,write} and so on (and the final PID needs to be sent back to the original parent) in order to set up and join all of the necessary namespaces.

In C, this code would be _immensely_ easier to read, write and maintain. Just look at LXC. Personally I really wish people had just gone with Rust earlier on rather than implementing everything in Go. I've had nothing but pain from Go.

[SwellJoe]

Thanks for explaining how it's done in runc. That does sound pretty awful. So, even though the initialization can be outsourced to a C function, you still would prefer to be working entirely in C? Are there no advantages to Go for runc? And, would it be possible for someone to write a somewhat standardized Go library for doing this grunt work?

Is it merely fear of C that keeps so much of the container infrastructure on Go? I've only spent a couple of weeks looking peripherally at Go, and I already like it better than C (which I've poked at peripherally for ~25 years), but I don't know it well enough to know its warts.

[sly010]

> Is it merely fear of C that keeps so much of the container infrastructure on Go?

Sort of. Go binaries are statically linked by default which is a must in situations where you are e.g. unsure about what libs are available in the current environment. You have to go through a lot of hoops to make sure your C executable is really fully statically linked.

[cyphar]

Actually Go binaries aren't statically linked by default anymore, and there are several hoops you have to jump through to make them statically linked.

Note though that Go packages will be statically linked into your binary by default.

[cyphar]

> you still would prefer to be working entirely in C?

No, but there are more options than just Go and C. Rust is an option that I'm shilling at the moment (though I've only started learning it, so take that with a grain of salt). The main reason I would want to write it in C is because there's a lot of string parsing code you have to write in order to make container runtimes work -- and as we all know that's probably the #1 source of security vulnerabilities.

> Are there no advantages to Go for runc?

There are, mainly due to network effect (everything else is written in Go) and getting contributions from the community (Go is easy to pick up). Unfortunately there are also disadvantages, and quite a few of those disadvantages are present in Go but are not present in other memory-safe and low-level languages (Rust is a good example because to be quite honest it's the only player in this space that doesn't try to do more than necessary).

Go is a good language for it's designed for (Web servers and similar things), but from my experience it's not the best choice for low-level tasks. We've seen cases where long-running container daemons (not naming any names) will crash if you run more than 1000 containers on a single system. They don't crash because of the actual daemon code but because of issues with Go's GC (it doesn't actually free memory sanely, it uses MADV_DONTNEED which inflates RSS and causes OOM to kill your daemon).

> And, would it be possible for someone to write a somewhat standardized Go library for doing this grunt work?

Of course (and you could argue that we have done that in runc with github.com/opencontainers/runc/libcontainer/nsenter), but the thing to note is that in order to get around problems in the Go runtime you have to split out a single piece of code into separate processes and have to now redesign how a single function would work. So moving it to a separate library means that development is even more frustrating (you've created an API around the internal implementation of whatever thing you're working on).

> Is it merely fear of C that keeps so much of the container infrastructure on Go?

I think the network effect is the main reason. Most of the people I've worked with know quite a lot of C (we do kernel work sometimes) so writing a runtime in C would be frustrating but entirely doable. The problem is that you couldn't just import it into a Go project (and people don't like cgo because it makes binaries harder to build in certain cases).

> I've only spent a couple of weeks looking peripherally at Go, and I already like it better than C

Go definitely has it's uses, and I still use it for new projects. For example I recently wrote a tool for dealing with OCI container images in Go[1]. The standard library of Go is quite nice (though I found some bugs in archive/tar but let's not go there) and I'm always quite amazed just how much you can do before you have to import external libraries.

But I recently started learning Rust and I am _really_ enjoying being able to understand what my program will actually look like when compiled. If you've ever had to strace a Go program, you'll know exactly what I mean. Debugging Go programs is basically fucking impossible.

[1]: https://github.com/openSUSE/umoci

[SwellJoe]

Thanks so much for the in-depth response!

I've been tinkering with go, and the project I'm planning to use it for is container-oriented (building/using/distributing them for non-technical users, more than working with them at a very low level like runC or similar, but still, it's very useful to know). As an aside, umoci looks, at first glance, like one of the components I thought I'd need to build...so, that's cool.

I'm not gonna keep bugging you with questions; I'll go read the code. (I'm also finding rust really neat, conceptually, even if I'm not yet finding it easy to read or understand.)

[jankedeen]

What a clumsy, terrible, crappy workaround. Congrats.

[cyphar]

I never said it was a nice hack. Personally I think this would be infinitely better in C or Rust.

Oh, and I haven't even mentioned the absolute shitfest that is the cgroup namespace and how you have to set up cgroups before you unshare it because its behaviour changes based on what cgroups you were in when you unshared it.