I found it a little funny that their big "win" for the nilness checker was some code logging nil panics thousands of time a day. Literally an example where their checker wasn't needed because it was being logged at runtime.
It's a good idea but they need some examples where their product beats running "grep panic".
Actually, if we were running into cases where we aren't logging a panic which is actually happening in production, then the first thing to note is that we need to improve our observability. The issue might or might not be recoverable, but it should be logged. If nothing else, it should show up as a service crash somewhere within those logs, which is also something that service owners monitor and get alerts on.
The advantage of NilAway is not just detecting nil panic crashes after the fact (as you note, we should always be able to detect those eventually, once they happen!), but detecting them early enough that they don't make it to users. If the tool had been online when that panic was first introduced, it would have been fixed before ever showing up in the logs (Presumably, at least! The tool is not currently blocking, and developers can mistake a real warning for a false positive, which also exist due to a number of reasons both fundamental and just related to features still being added)
But, on the big picture, this is the same general argument as: "Why do you want a statically typed language if a dynamically typed one will also inform you of the mismatch at runtime and crash?" "Well, because you want to know about the issue before it crashes."
Beyond not making it all the way to prod, there is also a big benefit of detecting issues early on the development lifecycle, simply in terms of the effort required to address them: 'while typing the code' beats 'while compiling and testing locally' beats 'at code review time' beats 'during the deployment flow or in staging' beats 'after the fact, from logs/alerts in production', which itself beats 'after the fact, from user complains after a major outage'. NilAway currently works on the code review stage for most internal users, but it is also fast enough to run during local builds (currently that requires all pre-existing warnings in the code to either be resolved or marked for suppression, though, which is why this mode is less common).
That makes sense. I hope my first comment came across as I intended, which wasn't as criticism of the product but a suggestion about how to talk about the value of the product.
No worries, that's how I understood it too :) Just adding a bit more context on why we feel the approach does beat "grep panic" for people checking out this thread.
Just tried this out on some of my own code and it nails the warts that I had flagged as TODOs (and a few more...). The tool gives helpful info about the source of the nil, too. This is great.
Building a type checker on global inference is the kind of thing that sounds romantic in academia - "no type definitions and yet get type checking!" - but ends up being a nightmare to use in practice.
Nilability of return values should be part of functions public interface. It shouldn't come as a surprise under certain circumstances of using the code. The problem of global inference is that it targets both the producer and the consumer of the interface at the same time, without a mediating interface definition deciding who is correct. If a producer starts returning nil and a consumer five levels downstream the call-stack happens to be using it, both the producer and caller is called out, even if that was documented public api before, just never executed. Or vice versa.
For anyone who had the great pleasure of deciphering error messages from C++ templates, you know what I'm talking about.
I understand the compromises they had to take due to language constraints and I'm sure this will be plenty useful anyway. Just sad to see that a language, often called modern and safe, having these idiosyncrasies and need such workarounds.
> Building a type checker on global inference is the kind of thing that sounds romantic in academia - "no type definitions and yet get type checking!" - but ends up being a nightmare to use in practice.
I really love Golang and how it focused on making the job of the reader easy. But with today’s modern programming language the existence of null pointer dereference bugs doesn’t really make sense anymore. I don’t think I would recommend anyone to start a project in Golang today.
With generics, can you not make a NonNil<T> struct in Go, where the contents of the struct are only a *T that has been checked at construction time to not be nil, and doesn't expose its inner pointer mutably to the public? I would think that would get the job done, but I also haven't really done much Go since prior to generics being introduced
Otherwise, since pointers are frequently used to represent optional parameters, generics + sum types would get the job done; for that use case, it's one of two steps to solve the problem. I don't foresee Go adding sum types, though.
Every type in Go has a zero value. The zero value for pointers is nil. So you can't do it with regular pointers, because users can always create an instance of the zero value.
This is one of those things which feels like just a small trade off against convenience for the language design, but then in practice it's a big headache you're stuck with in real systems.
It's basically mandating Rust's Default trait or the C++ default (no argument) constructor. In some places you can live with a Default but you wish there wasn't one. Default Gender = Male is... not great, but we can live with it, some natural languages work like this, and there are problems but they're not insurmountable. Default Date of Birth is... 1 January 1970 ? 1 January 1900? 0AD ? Also not a good idea but if you insist.
But in other places there just is no sane Default. So you're forced to create a dummy state, recapitulating the NULL problem but for a brand new type. Default file descriptor? No. OK, here's a "file descriptor" that's in a permanent error state, is that OK? All of my code will need to special case this, what a disaster.
Default gender male not how this works in practice. Instead, you define an extra “invalid” value for almost every scalar type, so invalid would be 0, male 1 and female 2. Effectively this makes (almost) every scalar type nullable. It is surprisingly useful, though, and I definitely appreciate this tradeoff most of the time.
(Sometimes your domain type really does have a suitable natural default value, and you just make that the zero value.)
This is a thread about Go, not about Rust. There is a bunch of interesting computer science in this post, and if interesting new computer science is a baby seal, Rust vs. Go discussions are hungry orcas.
One answer would be to provide something like a GetPointer() method which, if the inner pointer is nil, creates a new struct of type T and returns a pointer to it.
Without intention to offend. It's Golang, the language that famously ignored over 30 years of progress in language development for the sake of simplicity.
Partly this is out of memory of the good/bad old newsgroup days where this kind of thing somehow worked ok, until it didn't, but it definitely doesn't work on the sort of forum that HN is. We'd like a better outcome than scorched earth for this place.
Very interesting work.
I wonder what were the difficulties encountered.
Aliasing? Variable reassignment wrt short declaration shadowing?
Hopefully with time, when exploring union types and perhaps a limited form of generalized subtyping (currently it's only interface types) we'll be able to deal with nil for good.
It's really easy to check a field of a pointer struct without first checking the struct is non nil. Would be interesting if go vet or test checked this somehow.
Plenty of Go commentary in this thread but can I just say I'm glad to have learned about nilness? Suffered through a few nil pointer dereferences after deploying and having this analyser enabled in gopls (off by default for me at least) is a nice change.
I tried it and got a lot of false positives, but there wasn't so much output that I couldn't quickly pick out the interesting cases. This is very cool.
We'd be interested in the general characteristics of the most common ones you are seeing. If you have a chance to file a couple issues (and haven't done so yet): https://github.com/uber-go/nilaway/issues
We definitely have gotten some useful reports there already since the blog post!
We are aware of a number of sources of false positives and actively trying to drive them down (prioritizing the patterns that are common in our codebase, but very much interested in making the tool useful to others too!).
Some sources of false positives are fundamental (any non-trivial type system will forbid some programs which are otherwise safe in ways that can't be proven statically), others need complex in-development features for the tool to understand (e.g. contracts, such as "foo(...) returns nil iff its third argument is nil"), and some are just a matter of adding a library model or similar small change and we just haven't run into it ourselves.
In one case, it couldn’t tell that a slice couldn’t go out of bounds because I was iterating through it backwards instead of forwards. In another case, I had a helper method on a type to deal with initializing a named map type, but it couldn’t see that and thought the map was going to explode from being nil. Those are two false positives I remember off the top of my head. I can look it up again later.
Can this actually manifest? Even without the -race flag I think maps are a special case which will panic with a concurrent mutation error if access isn't synchronized.
> In the current Go implementations, though, there are two ways to break through these safety mechanisms. The first and more direct way is to use package unsafe, specifically unsafe.Pointer. The second, less direct way is to use a data race in a multithreaded program.
It's interesting that the 2010 article you linked suggests they might consider improving this but nope, Go 1.0 and the Go people use today just basically takes the same attitude as C and C++ albeit with a small nuance.
In C and C++ SC/DRF (Sequentially Consistent if Data Race Free) turns into "All data races are Undefined Behaviour, game over, you lose". In Go SC/DRF turns into "All data races on complex types are Undefined Behaviour, game over, you lose". If you race e.g. a simple integer counter, it's damaged and you ought not to use it because it might be anything now, but Go isn't reduced to Undefined Behaviour immediately for this seemingly trivial mishap (whereas C and C++ are)
Kind of ironic the raison d'être of Go is a memory safe language for concurrent programming but you can easily footgun yourself into doing something memory unsafe using concurrency...
Another example: thread A toggles an interface variable between two types, thread B calls a method on it. You can get the method of type X called with a receiver of type Y.
There are ways a Go program can fatal: by running out of heap, or stack, by corrupting variables by racing writes, by deadlocking, by misuse of reflect or unsafe, and so on.
I’ve seen it happen before because the stdlib actually directly just makes POSIX syscalls for a lot of things by default instead of using the native Go implementations and so you’re implicitly reliant on C code
I'm not sure if that was the best example to showcase NilAway. I understand there's a lot of context omitted to focus on NilAway's impact, but why is foo returning a channel to bar if bar is just going to block on it anyway? Why not just return a *U? If foo's function signature was func foo() (*U, error) {}, this wouldn't be a problem to begin with.
I have been thinking about this problem for a long time as well.
But I think that focusing on nils is a wrong analysis. The problem is the default zero-values dogma, and that is not going to change anytime soon.
Sometimes you also need a legitimate empty string or 0 integer, but the language cannot distinguish it from the absence of value.
In my codebase, I was able to improve the readability of those cases a lot by using mo.Option, but that has a readability cost and does not offer the same guarantees than a compiler would. The positive side is that I get a panic and clear stack trace whenever I try to read an absent value, which is better than nothing, but still not as good as having those cases at compile time.
No amount of lint checkers (however smart) will workaround the fact that the language cannot currently express those constraints. And I don't see it evolving past it's current dogmas unfortunately, unless someone forks it or create something like typescript for go.
It's not a dogma it's a breaking change to the language. Removing default zero-values is effectively a different language. Literally none of my code over the past several years (which otherwise all still works perfectly as-is) would work.
The Go team is very careful to avoid breaking changes (cue all the usual Well Actually comments regarding breaking changes that affected exactly zero code bases) and rightfully so. Their reputation as a stable foundation to build large projects upon has been key to the success and growth of the language and its ecosystem.
I have about a million and one other issues I'd like to see resolved first that don't involve breaking changes. It's a known pain point, the core maintainers acknowledge it, but suggestions to fundamentally derail the entire project are ludicrous.
Focusing on nils is fine. NilAway is fine. It's a perfectly reasonable approach and adds a lot of value. This solves a real problem in real code bases today. There is no universe wherein forking to create a new language creates remotely equivalent value.
I didn't said that we should remove default values, that is a wrong interpretation of my message.
For example we could have a new non-nilable pointer type (that would not have any default value), or an optional monad natively in the language (or any other thing in-between, there are many possibilities). That would allow the compiler to statically report about missing checks, without breaking backward compatibility.
But we all know that it's not going to happen soon because while not breaking any existing code, it goes against the "everything has a zero-value" dogma. That was the meaning of my message.
You are wrong regardless: there is no such dogma. There are numerous ongoing proposals discussing how to accomplish this. You're welcome to contribute. It took me a minute to find these proposals, as examples:
Sum types are a different issue (even if somewhat related) than what I am talking about.
Even if sum types were introduced, it would not help with nil values because - as you said - backward compatibility won't be broken.
If I had the luxury of spare time to contribute, I would probably spend it switching away to another richer language instead, because it would be cheaper, solve more of my problems and with a higher degree of certainty. And that's not even mentioning the attitude and toxicity of the community compared to most of other languages facing critics and ideas.
> Nil panics are found to be an especially pervasive form of runtime errors in Go programs. Uber’s Go monorepo is no exception to this, and has witnessed several runtime errors in production because of nil panics, with effects ranging from incorrect program behavior to app outages, affecting Uber customers.
Insane that Go had decades of programming mistakes to learn from but it chose this path.
Anyway, at least Uber is out there putting out solid bandaids. Their equivalent for Java is definitely a must-have for any project.
> Insane that Go had decades of programming mistakes to learn from but it chose this path.
Yup, every time I write some Go I feel like it's been made in a vaccum, ignoring decades of programming language. null/nil is a solved problem by languages with sum types like haskell and rust, or with quasi-sums like zig. It always feels like a regression when switching from rust to go.
True, and because of this, the language can be learned over a weekend or during onboarding, new hires can rapidly digest codebases and be productive for the company, code is straightforward and easy to read, libraries can be quickly forked and adapted to suit project needs, and working in large teams on the same project is a lot easier than in many other languages, the compiler is blazing fast, and it's concurrency model is probably the most convenient I have ever seen.
Or to put this in less words: Go trades "being-modern" for amazing productivity.
> It always feels like a regression when switching from rust to go.
It really does, and that's what I love about Go. Don't get me wrong I like Rust. I like what it tries to do. But I also love the simplicity, and sheer productiveness of Go. If I have to deal with the odd nil-based error here and there, I consider that a small price to pay.
And judging by the absolute success Go has (measured by contributions to Github), many many many many many developers agree with me on this.
It's funny how I always hear the point about new hires for Go. My team is a Rust shop at $DAYJOB that I created basically from scratch, so I had to onboard every new hire on the codebase. It's amazing how confident they are due to the compiler having their back, and how confident I am their code won't blow up that much in prod.
> code is straightforward and easy to read
I have to disagree. I don't want to read 3 lines out of four that are exactly the same. I don't want to read the boilerplate. I don't want to read yet another abs or array_contains reimplementation. Yes it's technically easy to read, but the actual business logic is buried under so much noise that it really hinders my capacity to digest it.
> the compiler is blazing fast
much agreed, that is my #1 pain point in rust (but it's getting better!)
> and it's concurrency model is probably the most convenient I have ever seen
this so much. this is what I hate the most with go: it pioneered a concurrency model and made it available to the masses, but it has too many footguns imho. this is no surprise other languages picked channels as a first class citizen in their stdlib or core language.
> Go trades "being-modern" for amazing productivity.
I don't think those two are incompatible. If we take the specific point of the article, which is nil pointers, Go would only have to import the sum types concept to have Option and maybe Result as a bonus. Would this translate to a loss of productivity? I don't think so. (oh and sum types hardly are a modern concept)
Also, there may be a false sense of productivity. Go is verbose, and you write a lot. Sure if you spend most of your time typing then yes you are productive. But is it high-value productivity? Some more concise languages leave you more time to think about what you are writing and to write something correct. The feeling of productivity is not there because you are not actively writing code most of the time. IIRC plan9 makes heavy use of the mouse, and people feel less productive compared to a terminal because they are not actively typing. They are not active all the time.
> Also, there may be a false sense of productivity. Go is verbose, and you write a lot. Sure if you spend most of your time typing then yes you are productive. But is it high-value productivity? Some more concise languages leave you more time to think about what you are writing and to write something correct. The feeling of productivity is not there because you are not actively writing code most of the time. IIRC plan9 makes heavy use of the mouse, and people feel less productive compared to a terminal because they are not actively typing. They are not active all the time.
This is my sense. "False sense of productivity" is an accurate statement - I've also found that it seems to be for a very specific (and not necessarily useful) definition of "productive", such as LOC per day.
It's not as bad as dynamic languages like Python, but very frequently Go codebases feel brittle, like any change I make might bring down the whole house of cards at runtime.
> It's funny how I always hear the point about new hires for Go. My team is a Rust shop at $DAYJOB that I created basically from scratch, so I had to onboard every new hire on the codebase. It's amazing how confident they are due to the compiler having their back, and how confident I am their code won't blow up that much in prod.
Same. Started a company, onboarded just about everyone to Rust. It went very well.
>It's amazing how confident they are due to the compiler having their back, and how confident I am their code won't blow up that much in prod.
I get what you're saying, and I'm glad you are having such a good experience with it. Disclosure, I am not talking down to any language here...in fact I actually like Rust as a language, even though I don't use it professionally.
I am just saying that Go is incredibly easy to learn, and I don't think there are many people who disagree on this point, proponent of Go or not.
> I have to disagree.
We'll have to agree to disagree then :-) Yes, the code is verbose, but it's not really noise in my opinion. Noise is something like what happens in enterprise Java, where we have superfluous abstractions heaped ontop of one another. Noise doesn't add to the program. The verbose error handling of Go, and the fact that it leaves out a lot of magic from other languages doesn't make it noisy to me.
> I don't think those two are incompatible.
Neither do I, but that's the path Go has chosen. It may also have been poorly worded on my part. A better way of putting it: Go doesn't subscribe to the "add as much as possible" - mode of language development.
> But is it high-value productivity?
Writing the verbose parts of go, like error checking, isn't time consuming, because it's very simple...in fact, these days I leave a lot of that to the LLM integration of my editor :-)
Is is high value? Yes, I think so, because I don't measure productivity by number of lines of code, I measure it by features shipped, and issues solves. And that's where Go's ... how do I say this ... obviousness really puts the language into the spotlight for me.
I honestly have started to wonder if the popularity of languages like Go and JavaScript are due to the lack of features in the base language. JavaScript in particular has had an incredible amount of effort invested in creating fairly limited, scattershot, and duplicative support for features that are "just part of the language" in Kotlin, or Rust, or even Java. It makes a really rich field for people who are interested in compilers, but did all this really make Uber a better company? Would all that effort have been better spent solving problems core to their business?
At my employer we have a pattern of promoting people who have done things like write a proprietary application gateway. The dev got a couple promotions and moved on to another company and we got stuck maintaining a proprietary application gateway with a terribly messy configuration and poor observability.
This is the core of the problem. Of course you can learn the language in a weekend, but you're bound to make the same mistakes developers have been doing for decades.
This may be ok, as you say, if you allow errors here and there because you are fine dealing with those problems. But at the other end, it may be a user that is affected by the error. Which may be ok as well, but why should it be? We lament the quality of software all the time.
Compare this to other engineering fields: unless you study the knowledge of those who came before you may not even be allowed to practice in the field. I would not want to use a bridge built by someone who learned bridge building in a weekend.
Software is different though, it's rarely a matter of life or death. Given that, maybe it's ok to not have the highest quality in mind, because the benefit of productivity far outweighs the alternative.
Go is just making a certain set of tradeoffs. If you try to fix all the "mistakes developers have been doing for decades", you get Rust. And considering that Rust is already Rust, there is not much point in trying to make Go another Rust.
The line has to be drawn somewhere. I think everyone has certain things they'd put on the other side of that line, and strict nils are probably at the top of the list for many, but overall it's good that the Go team is stubborn about not adding new stuff. If they weren't, maybe there would be better nil handling, better error handling, etc. but compiles would also get slower and the potential for over-engineering, which Go now discourages quite effectively, would increase. At a high level, keeping Go a simple, pragmatic language with a fast compiler is more important than any particular language feature.
The argument is that Go is making a _wrong_ set of trade-offs.
It was designed, specifically, as per Rob Pike, for _bad_ developers. Developers who couldn't be productive at Google because they weren't properly taught at unis [0].
Then it caught momentum and then here we are, discussing a bad language designed for bad developers as if there is nothing better we can do with our lives.
I don't think anyone is suggesting that Go should be like Rust. It's too late for that. We're suggesting that people should just use Rust (or Haskell, or F#, or any other robust functional programming language) instead.
> And judging by the absolute success Go has (measured by contributions to Github), many many many many many developers agree with me on this
You can't make up that other devs' opinions / preferences are identical to yours just because they use the same language, there are other important factors in play (e.g., if your company is using Go, then you'd be more productive in it and be more likely to choose to contribute in it even it Go is less productive as a language)
It could have done both though. It could have explicitly nullable types like Kotlin/C#. Or sum types like zig/rust/Swift. That wouldn’t make the language more complex to learn.
By definition, every bit added to a language makes it more complex to learn.
Sure, it could be done. Lots of things could be done to Go. The people who invented it are among the most brilliant computer scientists alive. It's a pretty sure bet that they know about, and in great detail, every single thing people complain Go doesn't have.
So every thing that is "missing" from Go isn't in it for a reason.
"Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away." -- Antoine de Saint-Exupéry, Airman's Odyssey
Not at all. Otherwise brainfuck would be the simplest language to learn. How do you currently represent a type that is A or B in Go? You have to use an interface. That’s much more complex than using a sum type would be.
It's definitely not as simple as "More features = harder to learn".
Removing footguns (nulls are a footguns, race-able concurrent APIs are a footgun) can make it easier to learn even though this may introduce new features (in this case sum types) to solve the problem.
This is a false dichotomy. One does not imply the other.
Go is also not a simple language. It is deceptively difficult with _many_ footguns that could have easily been avoided had it not ignored decades of basic programming language design.
Many things also aren't straightforward or intuitive. For instance, this great list of issues for beginners: http://golang50shad.es/
I'm sorry but nearly all of them are along the lines of "I came from language X and in X we did it this way, but Go's syntax is different". That's not a footgun.
You know what's a footgun? Uncaught exceptions popping up in places far away from where they were created at which point you have very little context to deal with it robustly. Use after frees. FactoryFactoryFactories.
I don’t have too much an opinion on either side here, but as a developer who works full time in Go (and has for >6 years) all these things exist in Go.
Uncaught exceptions -> panics, like what this nil catcher is aiming to solve
Places far away -> easy goroutine creation with no origin tracking makes errors appear sometimes very far away from source
Use after free -> close after close
FactoryFactoryFactories -> loads of BuilderFunc.WithSomething
Lots of other pains I could add that are genuinely novel to Go also, but funny that for everything you mentioned my head went “yep, just called X”
> I'm sorry but nearly all of them are along the lines of "I came from language X and in X we did it this way, but Go's syntax is different". That's not a footgun.
You're right, I meant to link that in reference to how Go can be difficult to learn despite how it simple it seems. Not sure how I a sentence.
The overview of that site explains its purpose/necessity quite well. Some things are footguns, many are just confusing time-wasters. Nevertheless, they are frustrating and hamper the learning process.
> This is a false dichotomy. One does not imply the other.
No it isn't, and yes it does. By definition, the more features I add to something, the more complex it becomes. So yes, Go achieves it's simplicity precisely by leaving out features.
> this great list of issues
I just picked three examples at random:
"Sending to an Unbuffered Channel Returns As Soon As the Target Receiver Is Ready"
"Send and receive operations on a nil channel block forver."
"Many languages have increment and decrement operators. Unlike other languages, Go doesn't support the prefix version of the operations."
All of these are behavior and operators that are documented in the language spec. So how is any of these a "footgun"?
> No it isn't, and yes it does. By definition, the more features I add to something, the more complex it becomes. So yes, Go achieves it's simplicity precisely by leaving out features.
More complex for whom? Not having generics made the compiler simple, but having to copy and paste and maintain identical implementations of a function (or use interface) adds more complexity for users.
Similarly, adding a better default HTTP client arguably makes Go more complex, but the "simple" approach results in lots of complexity and frustration for users.
> All of these are behavior and operators that are documented in the language spec. So how is any of these a "footgun"?
Perhaps I could have been clearer. I didn't mean that the entire list was of footguns, just that there are lots of confusing and unintuitive things beginners need to learn.
Some actual footguns off the top of my head:
- using Defer in a loop
- having to redeclare variables in a loop
- having to manually close the body of a http response even if you don't need it
> No it isn't, and yes it does. By definition, the more features I add to something, the more complex it becomes.
Yes, and go opts to include features that unnecessarily increase complexity in this manner, such as nil values.
> All of these are behavior and operators that are documented in the language spec. So how is any of these a "footgun"?
By this logic, no language with a spec can have footguns. C and C++, notorious for their footguns, both specify their behavior in the spec, so do they not have any footguns?
We’ve found that something like 50k+ LoC Go projects become impossible to maintain.
“Optimising your notation to not confuse people in the first 10 minutes of seeing it but to hinder readability ever after is a really bad mistake.”
— David MacIver
Go is a simple language that anyone can pick up in a weekend, but productivity plateaus once you’re doing anything that requires hard constraints or complex systems (the same is true for JS, Python, and other scripting languages).
As someone who is ~3 months out of uni, I don't really understand this amount of concern towards "new hires". I've interned at a place where I was thrown head-first into a industrial-grade Java codebase. The initial few weeks were rough. There were complex class hierarchies, patterns that weren't documented but had to be followed, interfaces from 10 different packages, really hard to grok names, magical DI, Lombok, complex mock testing all while 10 different linters and code coverage tools yell at you. But I cut through the complexity and finished my task in the end, not through any particular skill but by being stubborn and willing to learn.
None of the Google fresh hires I know personally are stupid. They are talented people who could be just as productive in a C++ or Java codebase. Maybe even better when you have features like Java's streams or C++ templates to throw at non-trivial problems. They might need more time, but it's something easily budgeted for. If new hires have to be productive from the first day, that's a problem the company has created and not the employee. If other languages have too many ways to do something, just enforce only using a few of them, teams have and continue to do that.
I use Golang in my current job. The library ecosystem seems fine. But even as a "new hire", the language frustrates me sometimes. Go's concurrency is "easy", but has a minefield of problems. Just off the top of my head, for-loop semantics [which to Golang's credit, is being fixed but it is absolutely a breaking change], just being able to copy a mutex by accident. These are bugs I've written and not had fun tracking down. In a year I'll have all these footguns memorized, but I could also have spent a year getting better at any other language. Even at my experience level, the Rust compiler gives me enough grief for me to know that when it's happy, whatever I've written will work. Nothing about Golang gives me that confidence.
> There were complex class hierarchies, patterns that weren't documented but had to be followed, interfaces from 10 different packages, really hard to grok names, magical DI, Lombok, complex mock testing all while 10 different linters and code coverage tools yell at you.
And this is exactly what Go avoids, in my opinion and experience.
Unless someone shoehorns Go into an Enterprise Java style (which, sadly, is possible, and sometimes done), the problems you listed with the Java codebase either don't exist in Go, or are orders of magnitude less painful to deal with, even in large Go codebases. Plus, the toolchain is pretty obvious, because most of it ships with the language.
And while my argument mentions new hires specifically, because the impact is most visible with them, this is just as important for mid and senior level developers; yes Go is sometimes more verbose (although enterprise Java and lots of C++ code still runs circles around Go in that regard) than it's contemporaries, it is also obvious. There is little magic, there is little action at a distance, and the opinionated style of the language discourages superfluos abstractions.
I have used quite a few languages so far in my career. Go is the first where I was able to comfortably read and understand std library code within the first week of learning the language.
> Just off the top of my head, for-loop semantics [which to Golang's credit, is being fixed but it is absolutely a breaking change],
It is technically a breaking change. Practically, it isn't, because there simply are no examples of production code in the wild that rely on this unintuitive behavior (As mentioned multiple times in the discussion on Go's issue tracker, the dev team did their research on that), and code that implements the (very easy) fix, will continue to work after the upcoming change.
> Nothing about Golang gives me that confidence.
My experience is different. I know that most of the problems the Rust compiler complains about will be handled by the fact that Go is GC'ed, and most of the rest I avoid by relying on CSP as my concurrency model (Can't accidentially copy a mutex if there's no mutex ;-) )
If a tool is any good I’m going to be using it for years. I’d much rather spend more time studying reusable ways the language can help me solve a problem with less code; that’s what’s productive. A quickly learned language is like a nearly empty toolbox.
From witnessing so many HN flamefests, between Go and Rust, it seems there are people who are genuinely more productive with Go than Rust, and vice versa. Not saying people usually lie when they claim to be more productive with one, but rather that their judgement is very subjective and not scientific. I do wish Go didn't have the data race bugs, though. It greatly weakens Go's "fearless concurrency" selling point for me. To me, Rust doesn't always make concurrency "fearless" in terms of complexity, but at least I'm not fearing actual memory safety bugs in a random library. There's unsafe, but I think the design tradeoff there is quite reasonable and workable.
Several language have "fixed the null problem" after the fact, though usually as an opt-in e.g. typescript, C#.
The problem with Go (for this specific issue, there are lots of problems with Go) is that they have wedded themselves extremely strongly to zero-defaulting, it's absolutely ubiquitous and considered a virtue.
But without null you can't 0-init a pointer, so it's incompatible with null-safety.
I think C# pretty much left the idea of everything having a default behind when they decided to fix nulls. Though obviously the better alternative is to have opt-in defaulting instead.
This is a good point. Null pointers are the billion-dollar mistake, but the real billion-dollar mistake is having "zero values" in your language. In addition to the problems with null pointers, zero values make loose constructor semantics like C# and Java tempting, where objects can exist in a not-fully-initialized state, leading to lots of room for confusing bugs. Without zero values to fall back on as a crutch, the language design is forced to tighten that up so that objects are either completely initialized or completely uninitialized, like in ML or Rust†, which is a much cleaner semantics. (The funny thing is, Go has the tools to get rid of zero values by virtue of not having constructors, but it chose not to use them for that!)
† Strictly speaking, objects can be partially initialized and partially uninitialized in Rust, but this is harmless as the borrow checker statically ensures that uninitialized fields of objects are never accessed.
While this is a great piece of engineering, and will certainly deliver a huge amount of value to any project, the fact that a whole new tool had to be built (and will have to be maintained) to address serious, fundamental shortcomings in the language is really quite sad.
The go version NilAway isn’t as good as the java version NullAway yet. But the team working on it is very responsive and eager to improve.
For java projects I think NullAway has gotten so good that it really takes the steam out of the Kotlin proponents. Hopefully NilAway will get there too.
It's worth bearing in mind that some of these runtime panics would have happened anyway even if the code had been implemented in (e.g.) Rust. Ugly real world code tends to make quite frequent use of unwrap() or equivalents. For example: https://github.com/search?q=repo%3Arust-lang%2Frust+.unwrap%...
Rust is a lot better in this aspect, but this is a symptom of not having proper code review and standards.
Do not forget that in some scenarios using unwrap is totally fine if a panic is acceptable.
The same could be said for javascript: How many time have we not wrapped JSON.parse inside a try catch? More than we would like to admit.
Really appreciate Rust “forces” you to handles all execution paths.
> Do not forget that in some scenarios using unwrap is totally fine if a panic is acceptable.
Looking through the first few pages, most of these panics are easy to audit, and are infallible or in contexts where it doesn't matter (internal tooling, etc). That's a pretty stark difference to every single reference being a potential landmine.
Yes, you are probably less likely to get a panic caused by a nil reference in Rust than in Go. My point is that the equivalent software written in Rust (or most other languages with option types) would probably have had at least some of these very same bugs.
Like, one of the first files has only .unwraps in the comments (like a dozen of them in a file), some are infallible uses, some are irrelevant-to-runtime tooling, etc.
But anyway, "some" is a lot smaller than "all". Just like some of memory safety issues would also have happened since you can still use unsafe in Rust, yet it's still a big step forward in reducing those issues in the ugly real world
It's a list of all instances of ".unwrap()" in the project, so of course it includes instances irrelevant to my point. Seems uncharitable to assume that I haven't looked through it on that basis.
My basis is the %, not the simple fact that it has irrelevant instances.
So let me charitably ask directly: have you looked through all the examples at least on the first couple of pages? And if you have, what % of instances is relevant to your point?
I think this is covered in my reply to shakow. Unless Rust is absolutely riddled with bugs, it’s obviously going to be hard to find uses of unwrap that are definitely bad. The point is that there’s no way to easily assure yourself that all the uses of unwrap are definitely good.
It would be similarly difficult to trawl through the source of the Go compiler and find definitely bad instances of pointer dereferencing. So does that mean that it’s not actually a problem in Go either?
The point is precisely that it's not always easy to figure out which instances are problematic.
If you think about it a bit, given that bugs are relatively rare in a mature project, it's going to be difficult to find a use of unwrap that's definitely bad.
I don’t see this as a band-aid. It’s doing proper type checking (static analysis) and that seems quite promising?
Getting good type errors without requiring type annotations seems like a win over languages that are annotation-heavy. Normally I’d be skeptical about relying on type inference too much over explicit type declarations, but maybe it’s okay for this problem?
This is speculative, but I could see this becoming another win for the Go approach of putting off problems that aren’t urgent. Sort of like having third-party module systems for so many years, and then a really good one. Or like generics.
I guess you could call it a bandage? The point wasn't to bad-mouth it or undersell it - bandaids are awesome. The point is that we need some external thing to patch holes in the underlying system.
"The Go monorepo is the largest codebase at Uber, comprising 90 million lines of code (and growing)"
Is this just a symptom of having a lot of engineers and they keep churning code, Golang being verbose or something else. Hard time wrapping my head around Uber needing 90+ million lines of code(!). What would be some large components of this codebase look like?
Imagine a multidimensional matrix with various payment methods, local regulations, cloud providers, third party dependencies, web/mobile platforms, etc. Then also add more dimensions for internal things like accounting, hiring, payroll, promotions, compliance, security, monitoring, etc. Then double it for Uber Eats or whatever.
There's a lot of overlap and some invalid combinations, but you're still left with a huge number of combinations where Uber must simply work. And every time you add a new thing to this list, the total number of combinations grows polynomially.
(Also, Go is slightly more verbose than most languages. I think that's a feature and not a bug, but it's one more reason.)
Dang, a little more verbose? Understatement of a lifetime. It's fine, if you like it not whatever, but it is quite a bit more verbose than many languages that I've used. My number 1 qualms go is with such simple building blocks requiring a bunch of redundant boiler plate. You're welcome to disagree with my opinion here.
A lot of people seems to gravitate toward languages with less dense cognitive load. I have learned to love kotlin, but its also a super dense set of syntax to power it's very expressive language.
Uber is famous for NIH syndrome. You can tell by their open source projects they've basically built every part of their infra from scratch. So it's not just the application code but everything else that helps run it.
I personally find uberFX to be a fantastic project. It isn't necessary for you to write golang with it, but it certainly does provide a great framework for organizing code so that you can ensure that writing tests is as easy as it can be.
Basically exactly this. Also, a lot of their "in production" open source projects are not "in production" but were generated and released as part of their broken promotion process.
When you have thousands and thousands of engineers, and they are evaluated by how much code they produce, and they need to justify their job and continued employmwment, you end up with a 90M line codebase.
It is the nature of large systems to grow. As software engineers we build libraries to build libraries, we build tools on top of tools to check our tools.
From what I've heard from ex-FAANG, I'd wager that a significant portion of the Go is code-generated for things like RPC definitions or service skeletons.
It amazes me that in 2023 this is not a solved problem by design of the language. Why go doesn’t adapt the “optional” notion of other languages so that if you have a variable you either know it is not null or know that you must check for nullness. The technology exists
I write a lot of Go and used to write a lot of Swift. Swift is what you’ll consider a modern language (optionals, generics, strong focus on value types), while Go is Go.
I appreciate both languages, and of course Swift feels like what you’d pick any day.
But, after using both nearly side by side and comparing the experience directly, I’ve got to say, I’m so much more productive in Go, there’s SO much less mental burden when writing the code, — and it does not result in more bugs or other sorts of problems.
Thing is, I, of course, am always thinking about types, nullability and the like. The mental type model is pretty rich. But the more intricacies of it that I have to explain to the compiler, the more drag I feel on getting things shipped.
And because Go is so simple, idiomatic, and basically things are generally as I expect them to be, maintenance is not an issue either. Yes, occasionally you are left wondering if a particular field can or cannot be nil / invalid-zero-value, but those cases are few enough to not become a problem.
This is a popular view, but, again, does not match my experience. I have only lead small teams (say 3 to 10 people) of either senior or very intelligent and motivated middles, but for those, limitations of Go are not a problem in any shape or form. Comparatively, we had significantly more mess (and debates) in Swift.
Try (the current incarnation of the ? operator) is actually a very clever trait which does rather more than that.
Types for which Try is implemented can Try::branch() to get a ControlFlow, a sum type representing the answer to the question "Stop now or keep going?". In the use you're thinking of where we're using ? on a Result, if we're Err we should stop now, returning the error, whereas if we're OK we should keep going.
And that's why this works in Rust (today), when you write doSomething()? the Try::branch() is executed for your Result and resolves into a Break or a Continue which is used to decide to return immediately with an error or continue.
But this is also exactly the right shape for other types in situations where failure has the opposite expectation, and we should keep going if we failed, hoping to succeed later, but stop early if we have a good answer now.
A big problem with Try is the function signatures...excuse me, I would like a <<T as Try>::Residual as FromResidual<Result<T, !>>::Output, please. Yes, that is a caricature and I don't know the proper signatures, but c'mon. Read the discussion for the Try v2 RFC if you want a better idea.
...and then they add more syntax sugar to partly sweep the complexity under the rug. I like Rust as much as the next person, but I'm apprehensive about how this will play out.
That won't work because in Go you often need to wrap errors with additional context.
I have worked with Rust Option/Rust types and found them extremely unergonomic and painful. The ?s and method chains are an eyesore. Surely PLT has something better for us.
There are several language design problems solved in the 20th century that Go designers decided to ignore, because they require PhD level skills to master, apparently.
Hence why the language is full of gotchas like these.
Had it not been for Docker and Kubernetes success, and most likely it wouldn't have gotten thus far.
speaking from personal experience, i selected go for a project because it is high perf, automatically uses all cores w/ goroutines, and is type checked
That's what the `func foo() (*T, error)` pattern is for. It's actually better than syntactic sugar for optional values because now you also have a descriptive reason for why the value is nil.
But if you really cannot afford to return more than one bit of information, do `func foo() (*T, bool)`.
Result<T,E> does this. I forget exactly why Result is actually different from, and in fact superior to, `func foo() (*T, error)` but IIRC it has to do with function composition and concrete vs generic types.
Result<T,E> is in one of two states: It either has value of type T, or error of type E.
(*T, error) is either T (non-nil, nil), or error (nil/undefined, non-nil), or both (non-nil, non-nil), or neither (nil, nil). By convention usually only the first two are used, but 1) not always, 2) if you rely on convention why even have type system, I have conventions in Python.
Leaving aside pattern matching and all other things which make Rust way more ergonomic and harder to misuse, Go simply lacks a proper sum type that can express exactly one of two options and won't let you use it wrong. Errors should have been done this way from the start, all the theory was known and many practical implementations existed.
I don't know much Rust, but wouldn't the analogy to (*T, error) be Result<Option<T>, E>, which has 3 states? Or is that not a common construct?
Because *T could be nil or non-nil, it seems like the analogy would be a nullable type in the Result<>. In Go, (T, error) would only have the states (non-nil, nil) and (non-nil, non-nil) if T is not a pointer. Still, the Result type seems better to me because the type itself is encapsulating all of this (and the error I guess cannot be null).
As others have mentioned, Result is a sum type so you either have a T or an E, there's no situation in which you can get both or neither.
The second part is that it's reified as a single value, so it works just fine as a normal value e.g. you can map a value to a result, or put results in a map, etc... , language doesn't really care.
And then you can build fun utilities around it e.g. a transformer from Iterator<Item=Result<T, E>> to Result<Vec<T>, E> (iterates and collects values until it encounters an error, in which case it aborts and immediately returns said error).
Don’t rely on half remembering how specific languages implement things, try and internalise the fundamentals. Go functions tend to return a tuple which is a product type, while rust’s result type is sum type. Product types contain. Both things (a result and an error) while a sum type contains a result or an error.
I was working on a Grab competitor, you would be surprised about the number of subsystems running there.
There are entire teams that are working on just internal services that connect some internal tools together.
There was also very little effectivity and efficiency in the era of cheap capital so there were tons of talent wasted on nonsense. Uber built their own slack for a while!! (before just going to mattermost)
People always ask who actually makes money on Uber... I think it's not the cab drivers, not the investors, who makes money is the programmers. It's a transfer of money from Saudis to programmers.
still, linux kernel is around 30 million lines of code if I'm not mistaken as a reference. most probably they have their reasons, but it smells weird to me.
It's can be difficult to understand by big companies write so much code, but it becomes obvious once you're inside of one: business software can be arbitrarily complex, because businesses can be arbitrarily complex. The guys in suits earn their paychecks by constantly coming up with new things the business could be doing.
"All" a kernel does (for some very large value of "all") is schedule userspace programs and manage the system's physical resources (memory, disk, devices). You can reach a point where a kernel is done, in the sense that it meets those basic needs with an acceptable level of performance. Kernel developers don't make extra money for every new feature they add - if the system is good enough, then it's good enough.
Microsoft Word is no small software. It's probably around 10 million lines of code.
As for "per locality business rules differ that's why so many lines of code.." seems like you can have a policy engine+DSL (JSON or YAML or custom policy language and engine) thus your code base shouldn't balloon to almost 100 million limes of code...
A lot of it will be location based. It has come up before here in the discussion of why there is so much in the app. They have to cater for all the different rules in every jurisdiction.
I don’t really buy the usefulness of trying to statically detect possible nil panics. In their example of a service panicing 3000+ times a day why didn’t they just check the logs to get the stack trace of the panic and fix it there? I don’t see why static analysis was needed to fix that panic in runtime.
What I would really like golang to have is way to send a “last gasp” packet to notify some other system that the runtime is panicing. Ideally at large scales it would be really nice to see what is panicing where and at what time with also stack traces and maybe core dumps. I think that would be much more useful for fixing panics in production.
There was a proposal to add this to the runtime, but it got turned down: https://github.com/golang/go/issues/32333
Most of the arguments against the proposal seem to be that it is hard to determine what is safe to run in a global panic handler. I think the more reasonable option is to tell the go runtime that you want it to send a UDP packet to some address when it panics. That allows the runtime to not support calling arbitrary functions during panicing as it only has to send a UDP packet and then crash.
I could see the static analyzer being useful for helping prevent the introduction of new panics, but I would much rather have better runtime detection.
As a language that’s focused on backward compatibility than features oriented this is the best and optimal way to reduce some of Go’s loopholes. The problem of using developer tooling to solve the innate problems is that they lack awareness
I do recommend the Go team to find a way to these tools to run before it complies, just doing go build while going through these tools first goes a long way than just using scripts
I found it a little funny that their big "win" for the nilness checker was some code logging nil panics thousands of time a day. Literally an example where their checker wasn't needed because it was being logged at runtime.
It's a good idea but they need some examples where their product beats running "grep panic".