[wmf]

"Asking for elevated permissions inside secure clusters at regulated financial enterprises or top secret government networks is at best a big delay and at worst a nonstarter."

But exfiltrating data with a userspace VPN is totally fine?

I'm also wondering why not use TLS.

[tptacek]

Every connection you make to a remote service "exfiltrates data". Modern TLS is just as opaque to middleboxes as WireGuard is, unless you add security telemetry directly to endpoints --- and then you don't care about the network anyways, so just monitor the endpoint.

The reason you'd use WireGuard rather than TLS is that it allows you to talk directly to multiple services, using multiple protocols (most notably, things like Postgres and Redis) without having to build custom serverside "gateways" for each of those protocols.

[eqvinox]

Adding your own network stack to bypass limitations like this works exactly until the point where someone notices that your userspace stack needs to fulfill the same requirements that the host stack does.

And then you're suddenly in a whole world of pain because all of this is driven by a stack of byzantine certifications (half of which, as usual, are bogus, but that doesn't help you), and your network stack has none of them.

(Written from first-hand experience.)

[Honeymari333]

Any tips I'm a beginner

[taeric]

I think the point was more that doing this as a way to avoid the red tape of getting permission to open a new connection is odd?

[tptacek]

I understand the impulse, but I think it misconstrues the "red tape" this method avoids. It's sidestepping a quirky OS limitation, which dates back to an era of "privileged ports" and multi-user machines. It's not really sidestepping any sort of modern policy boundary. For instance: you could do the exact same thing with WebSockets (and people do).

[taeric]

I was thinking websockets; though, I thought those largely hit the same criticisms? That is, tons of things moved to them specifically to avoid any firewall rules about what they were allowed to send over a network.

I'll fully grant that that seems to be the norm for everything browser related. Policies got difficult to install new software, just point your browser to this url and call it a day.

[mindcrime]

I was thinking websockets; though, I thought those largely hit the same criticisms? That is, tons of things moved to them specifically to avoid any firewall rules about what they were allowed to send over a network.

Arguably, this basic phenomenon has been going on for 20+ years. A lot of people by 2005-2007 or so had come to belive (and probably correctly) that a lot of the impetus for adopting SOAP based web-services over the preceding few years was simply because everything ran over ports 80 and 443 which were already open in the firewall. So deploying a remote service this way was more tractable than submitting a request to allow access to yet another port in firewall, and deal with the inevitable bureaucratic nightmare of getting that approved.

[Xelynega]

You can also sidestep that "quirky OS limitation" by just setting the first unprivileged port to 0(ip_unprivileged_port_start), no need for an new stack.

https://www.kernel.org/doc/Documentation/networking/ip-sysct...

[tptacek]

You can do that. Random programs cannot. Our CLI, which also does user-mode WireGuard and TCP/IP, doesn't even want to run under sudo. You're seeing the point, now: you want to build interesting network features that work the same way everywhere without demanding that your users be system administrators. Hence: user-mode TCP/IP.

[Xelynega]

I'm still not seeing the point sadly.

If you're running on a system you don't administrate that has ports under 1024 set as privileged, there's no way(with or without your cli) to have a userspace program receive TCP or UDP packets coming into the kernel from external devices for these ports(unless I'm completely mistaken).

What can you accomplish with "user-mode TCP/IP" that you can't from userspace with system calls?

[nonameiguess]

When they're talking about classified defense networks, the actual restrictions they mean is least privilege and separation of duties. Devs are not admins. They don't get root privilege on their machines. They can't create virtual network interfaces and they also can't change kernel settings. But if you put a full TCP/IP stack in userspace, well, they can run that and do whatever they want with it.

To answer the upstream question about why arbitary outbound connections are allowed, they're not. This is connecting to a cloud development environment, and I would have to assume this service can be self-hosted, because on a classified network, the "cloud" isn't the cloud as Hacker News readers know it. Amazon et all run private data centers on US military installations that only the military and the IC can access and they're airgapped from the Internet. If you're on a workstation that can access this environment, that's all it can access. The only place you can exfiltrate data to is other military-controlled servers.

[Xelynega]

If you're talking about developer machines, isn't the best(and easiest) solution to just run a VM that you administer so you can create virtual networks?

If you're talking about production machines, a userspace application wouldn't be able to sniff privileged ports without elevated permissions, so I fail to see how this application would let you get around that limitation.

[anyfoo]

Getting that change onto the system sounds like "at best a big delay and at worst a nonstarter".

[anyfoo]

You can't control what information flows through an outbound connection, not even in trivial cases. Even if you straight go ahead and say "I allow you to make this connection, but I'm not even allowing you to send any data", you have timing sidechannels to deal with. In any more reasonable case, an almost infinite number of things can be used to exfiltrate any data you want, even if you think you have not only full application-level inspection, but even application-level rewrite.

Pretty much the only thing you can do is somewhat filter out known-bad, not directly motivated outbound traffic, such as malware payloads with very clear signatures. This only works if it's "not directly motivated", because as soon as there's a person who wants to do it, they can skirt around it again.

[raggi]

fwiw, you technically don't need a privileged container to use tun, you just need suitable permissions on the kernel tun interfaces.

[tazjin]

Yeah, the optimisations are cool of course, but (maybe due to being unfamiliar with the tool?!) I didn't understand why they can't just `listen(2)`.

[vlovich123]

It’s answered in the opening paragraph although I’ll admit I’m still unclear.

> We are committed to keeping your data safe through end-to-end encryption and to making Coder easy to run across a wide variety of systems from client laptops and desktops to VMs, containers, and bare metal. If we used the TCP implementation in the OS, we’d need a way for the TCP packets to get from the operating system back into Coder for encryption. This is called a TUN device in unix-style operating systems and creating one requires elevated permissions, limiting who can run Coder and where. Asking for elevated permissions inside secure clusters at regulated financial enterprises or top secret government networks is at best a big delay and at worst a nonstarter.

The specific part that’s unclear is why encryption needs to be applied at the TCP layer and at that point if they need it at the transport layer why they’re not using something like QUIC which has a much more mature user-space implementation.

[dpeckett]

I think the key insight behind this approach (and I'm biased here having written something similar) is that the difference between QUIC and (wireguard + network stack) is A LOT less than you might think.

[Xelynega]

I'm confused on why they would need a TUN device for a client or server application, so why they would need this solution in the first place(even with their explanation).

As I understand the only reason you'd use a TUN interface is if you want to send/receive raw IP packets. Their marketing doesn't make it very clear what their product does, but I can't see a reason it would need to send/receive raw IP packets rather than TCP/UDP packets over a specific port...

[cricketlover]

Agree. Very unclear why they won't simply use a secure socket or why a user space tunnel will be needed.

I surmise that the reason might be that a user space tunnel might be faster (like maybe they can do UDP over TCP or something to gain speed improvements).

Good post nevertheless.

[immibis]

Or TLS. It seems to be a remote cloud desktop type of product, so why not use TLS like every other one?

[neonsunset]

The quote - is this yet another issue caused by abysmal FFI overhead in Go?

[zer00eyz]

https://www.reddit.com/r/golang/comments/12nt2le/when_dealin...

If your C doesn't fight the scheduler it isn't that bad.

[kbolino]

Great find! Specifically:

On a goroutine not locked to an OS thread (the default), don't take more than 1 microsecond in a single C call. If you need to take longer in C, lock the goroutine to an OS thread (runtime.LockOSThread), but then don't do things in Go that would park that goroutine (time.Sleep, blocking channel read/write, etc).

[tazjin]

There's nothing related to FFI calls in this quote.