[jerf]

I'm not 100% sure this is the case, but I believe the context of this goes something like this. As Go has added generics, there are proposals to add generic data structures like a Set. Generics solve almost every problem with that, but there is one conspicuous issue that remains for a data structure: You can iterate over a slice or a map with the "range" keyword, and that yields special iteration behavior, but there is no practical way to do that with a general data structure, if you consider constructing an intermediate map or slice to be an insufficient solution. Go is generally performance-sensitive enough that it is.

The natural solution to this is some sort of iterator, as in Python or other languages. (Contra frequent accusations to the contrary, the Go community is aware of other language's efforts.)

So this has opened the can of worms of trying to create an iteration standard for Go.

Go has something that has almost all the semantics we want right now. You can also "range" over a channel. This consumes one value at a time from the channel and provides it to the iteration, exactly as you'd expect, and the iteration terminates when the channel is closed. It just has one problem, which is that it involves a full goroutine and a synchronized channel send operation for each loop of the iteration. As I said in another comment, if what is being iterated on is something huge like a full web page fetch, this is actually fine, but no concurrency primitive can keep up with the efficiency of incrementing an integer, a single instruction which may literally take an amortized fraction of a cycle on a modern processor. With generics you can even relatively implement filter, map, etc. on this iterator... but adding a goroutine and synchronized commit for each such element of a pipeline is just crazy.

I believe the underlying question in this post is, can we use standard Go mechanisms to implement the coroutines without creating a new language construct, then use the compiler under the hood to convert it to an efficient execution? Basically, can this problem be solved with compiler optimizations rather than a new language construct? From this point of view, the payload of this article is really only that very last paragraph; the entire rest of the article is just orientation. If so, then Go can have coroutine efficiency with the standard language constructs that already exist. Perhaps some code that is using this pattern goroutine already might speed up too "for free".

The concerns people have about this complexifying Go, the entire point of this operation is to suck the entire problem into the compiler with 0 changes to the spec. Not complexifying Go with a formal iteration standard is the entire point of this operation. If one wishes to complain, the correct complaint is the exact opposite one, that Go is not "simply" "just" implementing iterators as a first class construct just like all the other languages.

Also, in the interests of not posting a full new post, note that in general I shy away from the term "coroutine" because a coroutine is what this article describes, exactly, and nothing less. To those posting "isn't a goroutine already a coroutine?", the answer is, no, and in fact almost nothing called a coroutine by programmers nowadays actually is. The term got simplified down to where it just means thread or generator as Python uses the term, depending on the programming community you're looking at, but in that context we don't need to use the term "coroutine" that way, because we already have the word "thread" or "generator". This is what "real" coroutines are, and while I won't grammatically proscribe to you what you can and can not say, I will reiterate that I personally tend to avoid the term because the conflation between the sloppy programmer use and the more precise academic/compiler use is just confusing in almost all cases.

[adrusi]

A goroutine really is implemented as a coroutine though. It's just that without this runtime optimization, there hasn't been a way to interact with them without letting a multithreaded scheduler manage their execution.

I don't remember where I came across this, but many years ago I saw python-style generators termed "semi-coroutines" which I'm a fan of. Python (frustratingly) doesn't implement them this way, but the beauty of generators is that by limiting the scope where execution can be suspended to the highest-level stack frame, you can statically determine how much memory is needed for the (semi-)coroutine, so it doesn't require a single allocation.

Zig takes that a step farther by considering recursive functions second-class, which lets the compiler keep track of the maximum stack depth of any function, and thereby allow you to allocate a stack frame for any non-recursive function inside of another stack frame, enabling zero-allocation full coroutines, as long as the function isn't recursive.

That would... probably be overkill for Go, since marginal allocations are very cheap, and you're already paying the runtime cost for dynamic stack size, so the initial allocation can be tiny.

I would love to see full proper coroutine support make it to Go, freeing users of the overhead of the multithreaded scheduler on the occasions where coroutine patterns work best. I remember back in 2012 or so, looking at examples of Go code that showed off goroutine's utility as control flow primitives even when parallelism wasn't desired and being disappointed that those patterns would likely be rare on account of runtime overhead, and sure enough I hardly ever see them.