[agentS]

> But you cannot write a non-blocking Go function in the same way.

The caller can make function "non-blocking" by wrapping the call in a goroutine themselves. (There's some subtle differences, but they are mostly irrelevant here). For this reason, I'd say there is (almost) no reason to introduce asynchrony in your API in the way you suggest. The rest of your post built on this example seems shaky to me, since it seems built on an example API that doesn't need to exist.

I'd say that "you don't have to care whether your code is async or not" is a overstating the case. I would append the qualifier "unless you're introducing concurrency". Considering that almost no low-level APIs are asynchronous, this usually happens rarely (or happens in low-level code like the HTTP server). Examples that have come up for me: making N parallel RPCs, writing a TCP server. In those situations, you care about async vs not.

In event-loop based systems, it seems like async is in my face all the time, even when doing things that are entirely sequential.

[Lukasa]

> The caller can make function "non-blocking" by wrapping the call in a goroutine themselves.

Sure, but if they want the return value then either they need to construct the Future-y wrapper I just described or they need to assemble it together in a collection of other function calls wrapped inside a function that itself is either Future-y or uses a long-lived channel to communicate results.

It is not novel to build up a non-blocking system from purely blocking method invocations. We've been doing that for years: it's called threading. Doing things this way has many advantages when written with appropriate diligence, and I'm not pretending otherwise. However, if you actually care about communicating between these arbitrary threads of execution than you either need Futures or queues (both of which are essentially just channels in Go), and at this point you've got the exact same problems as you get in NodeJS or any other asynchronous programming environment.

> The rest of your post built on this example seems shaky to me, since it seems built on an example API that doesn't need to exist.

I don't think that's fair: as I mentioned above, the fact that you as library author would not write the Future-y extension doesn't mean that the Future-y extension isn't built: you just force your caller to build it. That's fine, it's a perfectly good architectural decision (probably you should't be making those decisions for your user), but it doesn't remove the problem.

> I'd say that "you don't have to care whether your code is async or not" is a overstating the case. I would append the qualifier "unless you're introducing concurrency".

Sure. The thing that matters here is that Node is always introducing concurrency, because Node is concurrent. This is why all Node programs have to care about concurrency: they are all concurrent because their system is concurrent.

This is desperately inconvenient for many one-off programs, which is why I personally don't use Node for anything like that: I'd much rather use Python or Rust or Go. But that was never my argument. My argument was about OP's assertion that "with Go it doesn't matter if an operation is blocking or non blocking, that fact can totally be abstracted from the client code. Writing callbacks is tedious, promises are tedious, co routines with yield need plumbing and async isn't in the spec."

The first sentence is dangerously misleading (while technically true, any system that does that is usable only in that one context), and the second one misses the point, which is that those things get effectively built anyway in any moderate-scale concurrent system in Go.

But my biggest point is this: Go isn't magic in regard to concurrency, and there is a weird amount of magical thinking around Go. Go is a very good language with a lot to like, and I like it quite a lot. But when boiled down to it, Go's concurrency model is threads with a couple of really useful primitives. And that's great, and it works really well. But it's not new or novel.

The sentence "with Go it doesn't matter if an operation is blocking or non blocking, that fact can totally be abstracted from the client code" is equally true if you replace "Go" with "C", or "Python", or "Java", or any language with a threaded concurrency model. There's no magic here. It's the same building blocks everyone else is using.

[vishbar]

Go isn't just a threaded concurrency model, it uses an M:N greenthreads pattern. Also, when you say that Go I/O operations are blocking, it is true that they'll logically block a goroutine. However, under the hood, it uses the same libuv-style async IO (or IOCP on Windows) that Node does. An operating system thread doesn't get blocked; the goroutine is "shelved" and woken up again when the I/O is complete. It accomplishes the same kind of thing as Nodejs does, it just abstracts the async nature of the IO away from the programmer. I have to say I like it: procedural execution is easier to reason about.

[Lukasa]

Honestly, I think the distinction between M:N threading and straight OS threading is pretty minor. It grants some advantages to the language runtime: it can control the stack size, for example. But in terms of how it affects the development style and what kinds of bugs it encourages/discourages I don't think it dramatically differs from the OS threading model.

[sagichmal]

It is categorically different. OS threads are orders of magnitude more expensive, which makes them a nonstarter for most problems that are a good fit for lightweight conceptual concurrency.

[Lukasa]

It is not categorically different.

As I said above, green threading has advantages over OS threading, but they behave exactly the same in terms of design patterns and potential bugs.

This is what I was getting at when I said "not that different": compared to the difference between event-loop concurrency and threaded concurrency, M:N green threading is basically just a subcategory of threading.

[sagichmal]

No. Green threads enable entirely new classes of design patterns that are categorically infeasible with OS threads.

[Rapzid]

I believe that depending on the system call the thread handling the call could block, but it's not the same thread developer Goroutines are running on. Yeah though, same as nodejs.

[jerf]

The core problem here is that the Node community invented/popularized a connotation of "blocking" and "non-blocking" that is excessively event-loop-specific. The important difference in their connotation is that code that blocks blocks the whole OS process. The conventional meaning of the term referred just blocking the running thread.

In normal Go, nothing is blocking in the Node sense. (Oh, if you put your mind to it you can manage it, but I've never once encountered this as an practical problem, either in Go or the equivalents you can do in Erlang if you put your mind to it.)

This has profound changes on how you write code.

It's true, Go is not magic. It's just another threaded language in most ways, with the "real" magic in the community best practices around sharing by communicating instead of communicating by sharing. In theory, you could write an equivalent set of C libraries and get most of the same things, but you'd have a lot of library to write. (This is why things like porting C goroutines to C have a hard time getting traction. It can be done, but it's actually the easy part. Also, you'd still be in C, which is its own discussion. But you can get the concurrency.)

The real issue here isn't that Go is necessarily exceptionally strong at concurrency, the real issue is that Node is exceptionally weak. It introduces this new concept of "blocking" that only exists in the first place because it is weak, and then makes you worry about it continuously, to the point that many people seem to internalize the concept as what concurrency is, when it isn't. It's really just something Node laid on you. So when you step out of Node, and you see a community that isn't visibly as worried about "blocking" as the Node community, someone trained by Node thinks they are seeing a community that "isn't good at concurrency". My gosh! Look how cavalier they are about "blocking"! Look how they tell people not to worry about it, and how casual they are about having users wrapping library code in goroutines and explicitly telling library writers not to do the concurrency themselves. But what you're seeing is what happens when you simply no longer have the problems Node and "event-based code" brings to the table. Go is not magic in the general case, but, honestly, when someone coming from the Node world picks up Go, I can see why they might go through a period where they sort of think it is. There are really differences in code style, and how easy it is to write correct code.

You have to make sure you're not letting the limitations of one connotation of "blocking" spill over into the other, or you will have problems. (True in both directions.)

To speak to someone else's point, "futures" in Go don't "suck", they basically don't exist. If you're writing in a recognizably "futures" fashion, you are not writing idiomatic or even particularly good Go. You don't need futures, because (what are today called) futures are basically an embedding of a concurrency-aware language into a non-concurrency-aware language, and you don't need them when the language you're working in is already concurrency-aware. That's why you don't see futures in Haskell or Erlang either. (I have to qualify with "what are today called" because the term has drifted; for instance, Haskell does have explicit support for an older academic definition of the term with MVars, but modern software engineers are not using the term that way.)

[kasey_junk]

I've never in my life programmed in Node. When I say futures I'm talking about the logical concurrency primitive written about by Friedman/Wise in the 70s.

Lots & lots of idiomatic go code exists in that form (anytime you wrap a select that times out in a function you have a future).

Channels, what we are really discussing here, have 2 problems: the first is in abstraction, they don't provide basic primitives that other similar structures provide, like timeouts & cancellation. The second is in implementation. As futures you have to worry about all the edge cases around nil & closed channels. As queues they are highly contended.

[Lukasa]

I agree with this entirely. As I said many times before, I like Go and I like its approach to concurrency.

All I'm trying to do is to make sure that people who make bold claims about abstracting away blocking code aren't misleading others: when calling other code you should always be aware of how it interacts with the flow control of your program.

[philipov]

In your opinion, how does Python 3.5's async/await syntax compare to Go for writing concurrent programs? I work primarily with Python3 these days and have no Go experience.