[camdencheek]

Hi! Author here. Conc is the result of generalizing and cleaning up an internal package I wrote for use within Sourcegraph. Basically, I got tired of rewriting code that handled panics, limited concurrency, and ensured goroutine cleanup. Happy to answer questions or address comments.

[klabb3]

I literally started drafting my own structured concurrency proposal for Go 2 today, due to exactly the same frustrations you mention. Such a coincidence, and thanks for writing this lib. I will most certainly use it.

Please could you tell me if you have any thoughts on how to integrate these ideas into the language?

One thing I think should be solved (and that appears not addressed by your lib?) is the function coloring problem of context. I would really, really love if context was implicit and universal cancel/deadline mechansim, all the way down to IO. That way, only parts of the chain that NEED to use the context would be affected (which is a minority in practice). Any thoughts on that?

Finally, I think somebody should create a more visionary and coherent proposal for improved concurrency in Go2, because individual proposals on Go’s issue tracker are shut down quickly due to not making enough sense in isolation - and status quo bias. It’s a shame because I really love Go and want to see reduced footguns and boilerplate – especially with concurrency often being the largest source of complexity in a project. Please find my email in profile if you’re interested to pursue this further.

Thanks again.

[xyzzy_plugh]

> I would really, really love if context was implicit and universal cancel/deadline mechansim, all the way down to IO.

I don't think this is an improvement. Implicit behavior is difficult to identify and reason about. The only criticism of context that seems valid to me, aside from arbitrary storage being a huge antipattern, is that it was added long after nearly the entire standard library was authored, and it's usage is still relatively sparse.

We can agree that concurrency is difficult to use correctly, but since the introduction of generics it's much easier to wrap channels and goroutines.

Aside, in my experience if you're worried about boilerplate you're almost always looking at the problem wrong and optimizing for convenience over simplicity.

[klabb3]

It may not have sounded like it, but I agree with your philosophy entirely as the most important factor for language design. It’s also a big reason for why I prefer Go.

However, I also believe there are a few situations where explicitness should step aside. Go already breaks these rules for memory management - it has a GC, simply because it outweighs the benefits of explicitness. I argue that haphazard concurrency is comparable to manual memory management when it comes to both footguns and boilerplate, so I truly think it’s worth entertaining these ideas. That said, I can’t even back such a proposal myself, without first having it in hand and carefully studying it or even playing with it.

As for context, I’d like to have it trimmed down to ONLY cancelation and timeouts, possibly with user-provided errors (see Go1.20 cancel cause in exp). It shouldn’t have values at all. The reasons for the implicitness though, in order of importance:

1. Offer the caller to time out IO if the callee isn’t context aware or has forgotten to set a deadline. The majority of 3p library code I’ve seen makes such mistakes all the time, leading to socket leaks and the inability to tear down goroutines correctly.

2. To prevent the function coloring problem, which introduces substantial boilerplate duplication.

[preseinger]

It's a good package in general, save for the panic handling. Panics should not be handled in this way. Remove that wart, and it's solid.

[klabb3]

Is the alternative to crash and restart the whole process on panic? It would make sense if someone wants to write an in-process supervisor (similar to Erlang?) but this would be basically a main-wrapper - not a per-http-request thing (because Golang http itself would be corrupted).

I don’t know enough to say where the crash isolation boundary should best lie, BUT, assuming that you can catch panics at all, it makes a lot of sense that they are propagated upwards in the call stack. The idea of structured concurrency is that concurrent code is attached to the callers stack, at least in spirit.

[preseinger]

> Is the alternative to crash and restart the whole process on panic?

Yes.

> assuming that you can catch panics at all

A panic may happen to be safe to catch and recover from, but this isn't guaranteed, and can't be assumed in general. It's only safe to recover from a panic which you know is benign -- in all cases, across all build and runtime architectures. This is possible in packages that you fully control and which have no external dependencies, or (by fiat) in stdlib packages like net/http. But it's not the case for your service binary with a go.mod that's 100 lines long.