[raggi]

You can have multiple layers of security boundary on most of the customer-exposed surface area, and avoid more risky surface areas in the kernel.

Portable is a bit of a weird word here because for many of us with gray beards the word means architectures, kernels and systems, but I think in this context it tends to more mean "can run just as easily on my macbook as in a cloud container", but in practice the software isn't that portable, as Go isn't that portable - at least not in the context of vs. a niche C "portable network stack" that would build roughly anywhere that there's a working C toolchain, which is almost everywhere.

Constant security fixes for the kernel are a real pain in deployments unless you follow upstream kernels closely. If your business is in shipping Linux runtimes with a high packing density, you really need to find ways to minimize the exposed Linux surface area, or organize to be able to ship kernel upstream updates at an extremely high frequency (relative to normal infrastructure upgrade rates for kernels / mandatory reboots) (and I would not consider kexec safe in this kind of context, at all).

An alternative approach might be firecracker / microvms and so on, but those have their own tradeoffs too. The core point is that you want more than one layer between the host machines and the user code that wants to interact with Linux features.

[Xelynega]

> You can have multiple layers of security boundary on most of the customer-exposed surface area, and avoid more risky surface areas in the kernel.

I fail to see what "risky surface areas" in the kernel you're avoiding. You have more packets going through the kernel network stack(since you're wrapping a TCP connecting in a UDP connection that goes through the kernel) than just using the TCP stack in the kernel. Are you saying that the TCP stack in the kernel cannot be trusted, but a userspace kernel you maintain can(that's a bit ridiculous...)

> can run just as easily on my macbook as in a cloud container

Any POSIX C code that listens on non-privileged ports will run on machines with the correct glibc version(and you can statically compile the glibc or not need it like go does). This includes linux and macOS(and if you're using a library that's on multiple OSes you get even more support without having to implement TCP in userspace).

> Constant security fixes for the kernel are a real pain in deployments unless you follow upstream kernels closely.

I don't think you understand. You're still at the mercy of the kernel for security patches to the UDP stack, you're just now also having to maintain a TCP stack in parallel.

> An alternative approach might be firecracker / microvms and so on

Wouldn't an alternative approach just be to use cross-platform libraries and non-privileged ports?

> The core point is that you want more than one layer between the host machines and the user code that wants to interact with Linux features.

You just said the opposite... how can more things requiring security fixes be a bad thing, while you arbitrarily want more layers between you and the most security tested code for networking available to you.

[syzcowboy99]

> Are you saying that the TCP stack in the kernel cannot be trusted, but a userspace kernel you maintain can(that's a bit ridiculous...

Yes that’s exactly right. It’s not ridiculous. Netstack is written in a GC’d language which alone eliminates several categories of vulnerabilities that exist in the kernel. But more important than that is that it’s in USERSPACE. So even if you do compromise gVisor netstack the best you have is the capabilities that any other normal process has. Compare that to the kernel vulnerabilities where you potentially have cracked root.

> You're still at the mercy of the kernel for security patches to the UDP stack, you're just now also having to maintain a TCP stack in parallel.

The TCP stack is at least an order of magnitude more complex than UDP and has a correspondingly much higher number of bugs filed against it. Only relying on UDP is a security win.

[raggi]

> I fail to see what "risky surface areas" in the kernel you're avoiding. You have more packets going through the kernel network stack(since you're wrapping a TCP connecting in a UDP connection that goes through the kernel) than just using the TCP stack in the kernel. Are you saying that the TCP stack in the kernel cannot be trusted, but a userspace kernel you maintain can(that's a bit ridiculous...)

There's a constant stream of bugs in kernel network and IO interfaces, many of which require direct local interaction for exploitation, and aren't remotely attackable. Don't assume, spend a few hours and have a read through some.

> Any POSIX C code that listens on non-privileged ports will run on machines with the correct glibc version(and you can statically compile the glibc or not need it like go does). This includes linux and macOS(and if you're using a library that's on multiple OSes you get even more support without having to implement TCP in userspace).

That doesn't get anywhere near the use case here which is: run third party user supplied code unmodified.

> I don't think you understand. You're still at the mercy of the kernel for security patches to the UDP stack, you're just now also having to maintain a TCP stack in parallel.

The surface is not "UDP" and "TCP", this view is a huge distortion. As I suggested above, have a read through some of the relevant bugs over the last two years, and consider their implications in the relevant use case: running unmodified third party user code on a system.

> Wouldn't an alternative approach just be to use cross-platform libraries and non-privileged ports?

No, again, that doesn't meet the use case: run unmodified third party user code on the system.

> You just said the opposite... how can more things requiring security fixes be a bad thing, while you arbitrarily want more layers between you and the most security tested code for networking available to you.

Your characterization of Linux further suggests the exercise above would be a great experience.