[vbernat]

In Linux, many properties are task-related and therefore the API is also task-related. Namespaces are not the only problem. See for example https://github.com/golang/go/issues/1435. The problem is Go runtime does not provide a way to exclusively lock a system thread from existing or future routines. Most other languages would work just fine (C or Python for example).

[lunixbochs]

> The problem is Go runtime does not provide a way to exclusively lock a system thread from existing or future routines.

runtime.LockOSThread() does exactly that [1].

[1] https://golang.org/pkg/runtime/#LockOSThread

[cyphar]

The documentation for both GOMAXPROCS and runtime.LockOSThread lie to you. Neither of them allow you to stop the runtime from creating new threads (or executing library code in the wrong thread).

Trust me, we've been trying to get Go to co-operate with containers for quite a few years. It's not as simple as just reading the standard library docs. ;)

[lunixbochs]

Okay, I think I understand. I thought the problem referred to above with setuid was separate and would be fixed by locking the thread, but spawning sub-threads with the wrong UID is a problem. See also my sibling response.

[vbernat]

No, it doesn't. Go runtime can decide to spawn a new thread from the locked one. The new thread will inherit the characteristics (namespace, uid) of the old one. See https://github.com/docker/libnetwork/issues/1113 for more details.

[lunixbochs]

> Go runtime can decide to spawn a new thread from the locked one

That sounds like an implementation issue, why not assume the documentation is the intended behavior and this side effect is a bug? I'd support a CL to fix this behavior or add a block-clone-from-here runtime call (but the end result of that is you want the thread to exit when you're done with it, and not to go back into the pool... which is also new behavior). At the minimum, this behavior of new threads spawning from LockOSThread could be documented.

As a workaround, what about CGO -> pthreads -> spawn a control thread free from the Go scheduler -> call back into Golang to run a control loop function? You can do this in init() to ensure it has full control over itself. Or will Golang call clone() from unscheduled code?

[cyphar]

> That sounds like an implementation issue, why not assume the documentation is the intended behavior and this side effect is a bug?

Because after discussion with the Go devs they've concluded it's not a bug. To be fair, it's their decision to make a language runtime hostile to user's being able to mess with the process model, it just makes programs hard to write.

> At the minimum, this behavior of new threads spawning from LockOSThread could be documented.

The thing is it is documented[1], it's just very subtle:

> LockOSThread wires the calling goroutine to its current operating system thread. Until the calling goroutine exits or calls UnlockOSThread, it will always execute in that thread, and no other goroutine can.

An implication of the emphasised part is that if you use 'go' (or a function you call uses 'go') inside a locked goroutine, you are guaranteed that goroutine will be scheduled on another thread. Which is not what you might want or expect (personally I would expect goroutines created from a locked goroutine to act as normal coroutines). The problem is made much worse because a lot of the Go standard library uses 'go' internally and there's no way for you to know what functions use it and what functions don't (and what functions might use it in future versions). Not to mention that there are even more edge cases where functions you call could end up on separate OS threads.

> As a workaround, what about CGO -> pthreads -> spawn a control thread free from the Go scheduler -> call back into Golang to run a control loop function?

At that point you're really just massively hacking around the Go runtime. I would not be confident that such hacks would be a good idea in the long run. Remember that Go doesn't have any real forking model in its standard library or language, so the language provides no guarantees that it has to "play nice" with foreign threads.

Also, calling from foreign C code into Go is quite difficult (especially if you're calling into an _already running Go program_ which might reschedule your code at any time and would require hooking into core runtime internals).

> You can do this in init() to ensure it has full control over itself.

init() runs after the Go runtime starts up, you would want to do it as an __attribute__((constructor)) in C code so it is started before the Go runtime.

[1]: https://golang.org/pkg/runtime/#LockOSThread

[lunixbochs]

> language provides no guarantees that it has to "play nice" with foreign threads

I hope it continues to play nice, as I have a project depending on this behavior (bindings for a CPU emulator which spawns a thread for the CPU main loop, and calls back into my Go code on some events).

Thanks for the info. I guess the biggest problem is making sure `go` in a locked goroutine doesn't release threads to the scheduler from the locked goroutine? That's a weird thing to reason about. I initially guessed you'd need a separate feeder thread, but you want `go` from a locked thread to keep the uid and namespace of the locked thread.

So I guess there are two potential solutions:

- goroutines spawned from locked threads are pure coroutines and will not be scheduled on another thread

- goroutines spawned from locked threads can only execute on the locked thread, or on threads spawned from the locked thread that don't predate the goroutine. Child threads can also only be used for matching child goroutines, and will be more aggressively collected. This would converge closer to an oldschool threading model than green threads, but should hopefully prevent bugs like UID/namespace hopping.

This is the kind of thing you could possibly do with a small patch to the Go runtime. Go makes it so easy to build the compiler/runtime, perhaps forking Go for this is reasonable until there's an official solution.

There's also the issue of what to do with the primary thread that has been marked unsafe when it returns. Instead of LockOSThread, you probably want "give this OS thread and its children magic sandbox scheduling behavior, which includes collecting the thread when it returns, because we're going to call stuff like setuid".