- What is Go well suited for other than network programming?
- Why might one decide to write the backend API of their web app in Go, compared to say Grails, Python etc.
Other than some quite significant performance gains, I'd say the main upside for the case of a backend API would be channels. Concurrency is dead simple in Go - and if you want to do 5 different requests to ElasticSearch in parallel and merge the results when all of them are finished (like we do for Universal Search), that's just a few lines of very readable Go. Try that with a GIL, sure, multithreaded Python and Ruby is possible, but it's not for the faint hearted and not as easy to read.
Moreover, I love static deploys and cross compilation with Go. Compile your app (even to Windows!), copy it to a server, simply run it as a single binary. No dependency management, no apt-get & easy_install & pip, it just runs.
While Go does provide channels, I'd argue that they are not dead simple. I'm not saying this to bash Go, and I have willingly used it to solve problems. But I think it needs to be made more clear that this often-praised aspect of Go may disappoint those who are familiar with alternative techniques available in mainstream (read: not Haskell) languages.
For instance, look at the "Go Concurrency Patterns: Pipelines and cancellation" article: https://blog.golang.org/pipelines You'll notice the line "We introduce a new function, merge, to fan in the results" and after that, you will see how you have to write merge() yourself for every different data type that you use. Yes, you will have to repeat this same type of code, over and over, any time you want to pipeline, fanout, or merge, a new data type (unless you resort to interface{}). Furthermore, you will have to use the Go race detector to make sure you didn't actually mess something up.
I can't speak for Python or Ruby, but if you are using Node.js you can use a library like Bluebird which provides promise combinators. Then it's very easy to perform 5 requests and to handle errors and cancellation on one or more requests. You can do this and more on any arbitrary data type without writing merge() and nesting coroutine returning functions repeatedly.
So for handling async operations like dealing with APIs, I personally prefer tools like promise combinators or reactive programming (see Reactive Extensions for Javascript, also available in many other languages, or supplies in Perl 6 if you're crazy like me) over the significantly more manual approach of using typed channels in Go. I'm sure there are tasks where tight manual control of channels is important, but for the type of work I've been doing Go is simply too low level.
Generics can be nice, but they're clearly not "necessary" to any of the languages that don't support them. Besides, the cost that generics would exact in terms of syntax complexity, compile time, and startup time is something that many Go-detractors dismiss as irrelevant.
Myself, I think enforced tab-based formatting is utterly insane, but ultimately it doesn't matter. Part of the philosophy behind Go is to keep certain things very simple, and leaving generics out is a big part of that. I don't think that they're going to change their mind because non-Go programmers advocate for them. If that makes it the wrong language for your project, then there are so many others to consider.
But Go already has generics: channels, maps, make(), len(), etc. are work on generic types. You couldn't have typed channels without generics!
Go just doesn't have any syntactical way of declaring generic types. But given the above, nobody can argue that generics isn't extremely useful, even in the context of Go. "Oh, but the official library is special because it's part of the language", someone might argue. But one would have to be fairly damn obtuse to claim that Go would be better without the built-in generics, or that for some reason those benefits wouldn't extend to the language as a whole, if implemented.
I'm really surprised the authors of Go decided to allow generics only for the official library, because they've had to jump through some serious hoops to avoid it. If you look at "reflect", "builtin" and "sort", to pick a few, those packages are a graveyard of typing awkwardness. Look at the sorry state of the sort package, which even has a special function to sort strings. It goes on and on; every time I work with Go code, I end up implementing functions like min() and max() and cmp(). Why is "range" special? Why isn't there a way for generate iterators for any value? Etc.
Go is "simple", sure, but ends up being rather complicated as a result, with tons of the same code having to be written over and over for different types, and tons of typecasting between interface{} and real types, and so on. Nobody (as far as I can see) is asking for Haskell-style typeclasses or operator overloading or type traits or higher-kinded types or any of that. Several up-and-coming languages (such as Nim) implement generics without going overboard with complexity; quite the opposite, generics makes those languages simpler.
>Go is "simple", sure, but ends up being rather complicated as a result, with tons of the same code having to be written over and over for different types, and tons of typecasting between interface{} and real types, and so on. Nobody (as far as I can see) is asking for Haskell-style typeclasses or operator overloading or type traits or higher-kinded types or any of that.
Exactly. We're not asking for much here, just the ability to do the most basic kind of type parameterization.
The only benefit afforded by missing generics is simplicity in the compiler. It does nothing or worse than nothing when it comes to simplicity in actual Go code.
Even if generics never come to Go, my hope is that at least common pipelining tasks like merge() will become part of the stdlib and have magical generic support... but I haven't heard anything to indicate that such a thing will happen. :(
Python has good co-routine support. In Python 3.5 we have async/await and the concurrent.futures module. For I/O bound tasks like running REST APIs or MapReduce jobs I don't see the GIL being much of a problem; you spend eons waiting to do a few milliseconds of work then wait some more.
> While Go does provide channels, I'd argue that they are not dead simple.
I'd agree.
In my experience it's easier to explain a solution to a fundamental problem than to an abstract one. Channels, futures, promises... all very abstract concepts; useful to the cognoscenti but none are satisfactory at solving the fundamental problem of parallel execution. Hence everyone in their camps about which is right for which tasks.
So even with channels parallel programming is still difficult. You just have the added burden of a different, unique abstraction. Every language ecosystem either has their own community-adopted one or a plethora of them.
I think I'll reserve, "dead simple," for when we have a universal language of parallel execution. Until then... we don't know how to compute!
> Concurrency is dead simple in Go - and if you want to do 5 different requests to ElasticSearch in parallel and merge the results when all of them are finished (like we do for Universal Search), that's just a few lines of very readable Go. Try that with a GIL, sure, multithreaded Python and Ruby is possible, but it's not for the faint hearted and not as easy to read.
This example is terrible. Waiting for DB responses, ElasticSearch queries, and long running IO is one place where Python and Ruby multi-threading with a GIL works great. A GIL means multiple threads can't execute Python code at the same time. All of those tasks are by definition NOT running Python code, they're sitting around blocked waiting for responses.
A better example would be something like image processing, where an image is loaded into memory, broken into multiple independent chunks, and each chunk is processed at the same time in multiple threads. In Go that should work just fine, but in Python and Ruby each thread will spend a lot of time waiting for the GIL.
>Other than some quite significant performance gains
I am aware that other languages perform better in benchmarks than, say, Python does. But in my experience, I've not ever found the speed of the language to be a bottleneck when I'm benchmarking and optimizing for scalability in a web app.
It's always something else. The database interface, the network, a crappy web framework, whatever. It always seems to be something other than the fundamental language that bogs things down.
I'll openly admit I might be missing something or that perhaps I haven't tried to scale high enough. I just don't get how it's relevant that x language is y times faster than Python when Python hasn't ever been the problem. There's always just so much more low-hanging fruit than the language.
> Try that with a GIL, sure, multithreaded Python and Ruby is possible, but it's not for the faint hearted and not as easy to read
Actually that example does tend to be simple and readable in Python and Ruby. I've used a pattern similar to this to parallelize calls to a few back-end services from a Ruby app and it worked out great. The code I used looked something like this: https://gist.github.com/kevinmcconnell/8365521, which I think is quire readable.
I've definitely found Go's approach to concurrency to be very helpful in other situations; I just think that example could be a bit misleading.
> Try that with a GIL, sure, multithreaded Python and Ruby is possible, but it's not for the faint hearted and not as easy to read.
As others have noted, the example use case is one where multithreading with a GIL/GVL isn't particularly problematic. Moreover, both Python and Ruby have GIL/GVL-free implementations (in Python's case, Python 2 only in Jython/IronPython; in Ruby's case, much more current in the language level supported, since current JRuby is Ruby 2.2-compatible.)
I'm confused: a backend for a web app's API is a "networked" program.
With Go (as compared to Python), it's super easy to define some structs, serialize and deserialize them as JSON that you check, and send them to the client quickly and efficiently.
The more annoying question for web apps (and productivity in general) is the static (+-strong) vs dynamic (+-weak) typing. Go, like Scala, does a great job inferring types (IMO) letting you feel fairly productive, but some people will always yearn for the "anything goes" dynamism (sadly combined with the lack of compile-time checking that would save you on larger projects).
Apart from the already mentioned reasons, another one would be maintainability.
Go was designed with this in mind and some of its shortcomings come from this approach. It is a relatively small language, with one way of doing a certain thing and a mandate of using standard libraries and strict code formatting.
It sounds limiting but it is productive and makes every Go developer write in the same style.
So whether it is you that you return to your project after a few months or a new developer, you will be able to (re)engage with the code quickly.
This is one of the reasons I'm pushing Go at work.
I've had to maintain our current apps which have received very little maintenance work during the years and I wish Go would have been present when they were made.
> What is Go well suited for other than network programming? Why might one decide to write the backend API of their web app in Go, compared to say Grails, Python etc.
You'll get a lot of different opinions on this. I'll just speak based on my experience, since Python was my primary language before coming to Go.
Everything I used to use Python for, I can do faster in Go. The notable exception to this is statistical analysis, as Go does not have any FORTRAN bindings[0], whereas Python does (through Numpy & co.). I still use a combination of Python, R, and other languages for this.
I came to Go for the static typing and native concurrency[1], but I stayed because I'm more productive in it. Python and Go are about equally fun to write, but because I can build things much faster in Go, it ends up feeling more rewarding overall, given that my time is limited.
If you don't really have a pressing need for anything else, you can stick with Python. But I found Python to be slow to develop in and cumbersome, and after trying Go out, I realized I was 100x more productive in it.
[0] apparently this may no longer be true; if so, that's exciting news!
[1] this was also before Python had built-in async. I still prefer Go's concurrency model and syntax, but at least Python has this as an option now.
Everything I used to use Python for, I can do faster in Go. The notable exception to this is statistical analysis, as Go does not have any FORTRAN bindings, whereas Python does (through Numpy & co.).
Coming back to the main question. I use Go as my primary language these days. E.g., my dependency parser and neural net dependency parser (which uses the aforementioned BLAS binding) are written in Go:
What I like about Go: it's C-like without the unsafety of C nor the complexity of C++. Moreover, I've found that working in Go is generally as productive as Python (short compile times, good tooling, completion in vim, lightweight package system), while being much faster and better-fit for large projects.
What I dislike about Go: it's a cliché, but the lack of parametric polymorphism is jarring.
Assuming you need float64, BLAS and LAPACK are most easily accessed through the wrapper packages godoc.org/github.com/gonum/blas/blas64 and godoc.org/github.com/gonum/lapack/lapack64. This way code can be written to either use the native go implementations or the assembly/c/fortran ones.
100x? Even 10x is very significant. Why is it that you feel so much more productive in Go? Is it ease of refactoring? Lack of frustrating bindings bugs? Something else?
Yeah, I know 100x is a huge multiplier, but I've actually tracked myself to the extent possible, and that really is the right order of magnitude for me.
There are a number of reasons. I find refactoring is way, way simpler. Refactoring in Python feels painful enough that I always put it off until it gets absolutely necessary. With Go, I find it takes way less time, and also less mental energy.
The static typing and strict compiler (enforcement of imports and lvalues, etc.) works well for my writing style. I write what's in my head, without worrying about the small details (only the high level logic), and I can be confident that it'll be easy to fix the small details (syntax, typos) afterwards.
It's kind of like how writers often work - they dump words on a page, focusing on getting the main points across, and it's the editor's job to make sure they fix the grammar and style. I end up having a conversation with the compiler and when it stops telling me anything, the code usually does what I want it to.
I agree that static typing at least for me gives massive reduction in time in medium to large projects. And despite what many think about Java I am able to be a multiplier faster than Python with it (I would say as much as 4x at times).
What I don't understand is all these people claiming that the language is slowing them down by massive factors. I guess I'm either incredibly stupid or people on HN are incredibly smart (probably both) but I just can't even think fast enough for the syntax of the modern languages to really slow me down (ignoring copy n' paste exceptions and build time issues). For example to create the mental model of a dumb video game I'm making is taking more time than the actual coding.
In fact I would say if anything slows me down its bugs and/or features missing in immature open source libraries, crappy tooling, and lack of documentation and most importantly not fully understanding the problem (or what to create in terms of video game). And I can say this definitively about Rust... the language is awesome but I'm slow as crap in the language because of random stuff breaking and lack of good libraries.
I'm not saying Go has the above issues (on the contrary it now is rather mature) but I have hard time believing the 100x (and that is my opinion :) ) of going from problem to fully coded solution.
The static typing and strict compiler (enforcement of imports and lvalues, etc.) works well for my writing style.
It seems you more enjoy just "Not Python" than Go itself.
Go is still pretty awful and designed without really consulting what would help users. It's just designed for what the creators want, but now millions of people are trying to use it—not just 8 people inside the Nation of Google. It's getting worse for the average developer as time goes on. But, one thing developers love doing is understanding broken things, so in a away, the more difficult a system, the more nerds like it because it gives them accomplishment and the ability to exclude non-understanders. (Plus, Tabs? Tabs? In 2015? Is the Go development process run by monkeys living in Antarctica?)
> It seems you more enjoy just "Not Python" than Go itself.
No, I have experience with lots of languages. I'm comparing to Python here because that's what OP asked for.
I don't think Go is "just designed for what the creators want", but even if it were, I don't care, because that's what I want as well.
I'm talking about my personal experience of Go based on my own experience writing Go full-time for over three years, which is as long as the language has had a stable release. With all due respect, it's highly unlikely that a review from someone using it for four days is somehow going to change how productive I've already found the language makes me.
With all due respect, it's highly unlikely that a review from someone using it for four days
But, the review goes into a rant about how the Go community just thinks it's the greatest and refuses to listen to outside opinions due to a sense of inbred and ungrounded superiority... kinda like what you just did there.
A very significant reason why I love Go is that it fits in my head. I can use all the keywords, constructs, all the builtin functions effortlessly, while the standard library is a breeze to work with.
If you don't like LOC, then what sort of interpretation are you giving to "100x more productive"? I'm not so sure LOC is way off.
But in any case, if a person says he is "100x more productive" in one language than another, then I expect he can complete in 1 to 7 days what would take 100 to 700 days (i.e., 3 months to 2 years) in the other language.
Strange that I have to even highlight the obvious, but anyone who says they're 100x more productive in Go than Python is exaggerating to an extent that I find it hard to trust anything they say.
Again, "lines of code" is a shitty metric. Functionality might be one, customer support might be another... but even if you choose LOC, 100x is not impossible or even unlikely. When learning Go, I went back to undergraduate coursework, and picked some example problems from an advanced programming class. I solved these problems with Go.
Seventeen years ago, I dropped that class, because it was taught in Java. According to pure LOC, I'm something like 10,000,000 times more productive in Go than in Java, because Java was so gross I chose to withdraw from the course rather than waste my time rolling in mud.
Perhaps, setting LOC aside, a given programming languages matches a developer's thought habits better. In such circumstances, not only does she finish the nominal task quicker, she is able to respond more quickly to QA feedback, or to changing requirements, or to other business considerations. Perhaps a different paradigm enables her to foresee shortcomings in an existing design. These things not only improve one developer's productivity; they make the entire team more productive.
There is so much more to programming than "how many lines of text did you shit out today" and it is incredibly naive to behave as though LOC is the prime metric. We, as an industry, have been aware of this for at least forty years now. It's time to stop.
It's difficult to write a program without writing lines of code.
There are dumb measure, smart measures, and dumb ways of applying smart measures. We could consider a little intellectual generosity, that intentions were well-met, and not that a 50 year old used car salesperson turned programing manager was irrationally demanding X lines of code per day.
- The language is simple. No need to lookup strange keywords or try and figure out too much cleverness in the language itself. It promotes readable code.
- A basic and good type system. I don't have to worry much about that int turning into a string.
- Strong default libraries. No need to rely on third party libraries to do a lot of the common API things.
I've used Go for neural networks and machine learning with great success. Fast, safe, fairly straightforward. Much easier to parallelize than with Python.
I wrote some scientific code in it (some parsing, some processing). It was much easier to ensure that the code was secure (important when you parse possibly evil files), compared to C++, and it was much faster than Python and pretty easy to maintain.
Situations where you want a modern, strongly-typed language with easy C bindings.
IO-heavy operations, especially where it's primarily IO bound but there is a decent computational aspect (eg, working with an embedded database or doing lots of flatfile munging).
I've found the place where Go crashes and burns is if you're trying to implement / use lots of sophisticated custom data structures. It really really wants you to use the builtins, and the lack of generics is irksome.
We use golang for a lot of our ops stuff. Earlier it was a repo of a mash of different python/ruby scripts. The runtime dependencies were hard to keep track of and they had to be installed on all the machines depending on the script being run.
With golang it's just one static binary that is scp'ed across the nodes. This easy of deployment isn't talked about much.
We were also able to go further and put a RESTFUL API on top of these tools using negroni and gorillamux.
If you are interested in an approachable API for CSP, Go is good, along with Clojure's core.async. If you want to write asynchronous code, they are both great leaders in that area right now. If you don't need that API, I personally find the reasons harder to find in favor of Go.
I think Go is a great language for sophisticated scientific computing. Go scales well to complex code bases, and thus it is a great language for building an ecosystem. The tools are not all there yet, but the foundations are being built github.com/gonum
Moreover, I love static deploys and cross compilation with Go. Compile your app (even to Windows!), copy it to a server, simply run it as a single binary. No dependency management, no apt-get & easy_install & pip, it just runs.