I hate to be "that guy", but you should try Rust. It also has amazing tooling, and is probably more comparable to C/C++, considering you don't have to use a garbage collector in Rust.
I really like Rust, but in my mind it's still relegated to the periphery of tasks that must be super fast (like C/C++, to your point) and/or low-level. Rust has made impressive strides at reducing the toil involved in compile-time memory management, but there is still a big productivity gap between the borrow checker and GC.
Frankly, people are making boatloads of money off of software written in Python and JS--languages which are both far less safe than Go and far slower and yet Go is on par with those languages with respect to productivity (I argue it's more productive due to its static typing features, but others argue it's less productive presumably because of the learning curve). Most software doesn't need to be so fast or correct.
Also, I work in distributed systems and SaaS (software as a service). In this world, most errors are silly type errors ("undefined is not a function", forgetting to `await` some async function, etc) that would be caught with a flat-footed type system like Go's. Beyond that, the next largest bucket are issues that even a sophisticated type system like Rust's wouldn't help with, such as infrastructure issues (missing/misconfiguration of some cloud permission, networking rule, etc), misconfiguration of the service, race condition with many processes accessing the same networked resource, deployment error, etc. This means that Rust's stated advantages aren't really as nice as they initially sound.
As a programmer, I can appreciate a super fast language with a strict type system; however, as an engineer or a technologist, Rust is rarely a good fit for me. Go seems to be a lot closer to the sweet spot, which makes sense because it was developed expressly with distributed systems in mind.
> In this world, most errors are silly type errors
I can’t speak to python, but typescript has basically fixed this problem overnight in the javascript ecosystem. The typescript compiler finds almost all small bugs like this while I’m coding. And as an added bonus, type hints allow the IDE to be much more helpful - adding to jump to function support, autocomplete, method parameter suggestions (or documentation on hover). And typescript is easier to read than javascript because you don’t have to guess what data type some variable is.
I love javascript’s quick and dirty nature, but I still use typescript instead of javascript now for any code I write that I expect to survive the week. (And as a bonus: you still get IDE type hints when calling typescript functions from raw javascript!)
Typescript’s type system is also more powerful than Go’s. It supports enums, genetics and type unions (eg x: string | number).
Go sits in the awkward place of having a worse type system and no significant advantages over typescript for me. It’s awkward to use go on the web. Go is faster than javascript on the server but that usually doesn’t matter. And when it does I can reach for C or Rust. Both of which work really well with native code, or with JS through wasm.
Rust is much harder to learn than Go - but personally I’ve climbed that hill already. Once you’re over that hill, Rust is much more expressive. On purpose both ways - go isn’t trying to be expressive, and rust is. I love rust’s parametric enums and output types in traits. Which I now sorely miss in other languages.
I can imagine Go shining a lot more brightly when working with a team which has mixed skill levels. Most of my work lately has been solo, so I don’t need gofmt to enforce a consistent code style, or anything like that. I do miss Go’s green threads. Rust’s afterthought scattergun approach with threads and futures feels like a mess. But I can’t see myself ever really using Go. It’s weak in areas I want it to be strong (eg the type system). And strong in areas I just don’t care much about. (Eg consistency).
Rust is very expressive, but it also makes some tasks that are relatively simple in other languages much harder (anything involving tree structures, for instance), and some of what it gives you expressive control over is stuff you usually don't need to think about at all in other languages.
Rust can do some things Go simply isn't suited for right now; for instance, there is no way we're getting kernel Go in the foreseeable future. You couldn't reasonably build a browser, or browser components, in Go.
Both languages have their place. There is a lot of overlap, and the original comment about how one might prefer Rust if what they miss most in Go is generics makes sense. But the idea that the only thing you'd gain by choosing Go over Rust is easier development on big teams is just false. One language is GC'd, the other is borrow-checked. GC makes a lot of things faster to build. This debate ended in the mid-1990s.
> Rust is very expressive, but it also makes some tasks that are relatively simple in other languages much harder (anything involving tree structures, for instance)
Much harder, not really. Rust lets you add mutability, reference counting and thread safety to your data structures, you just need to know when to use these features. Yes, Go has fully general GC but very few problem domains have a real need for general GC.
> Yes, Go has fully general GC but very few problem domains have a real need for general GC.
No, no one needs a GC, but many problem domains need productivity, and GC is the most productive way to manage memory to date. As previously mentioned, Rust has made truly impressive strides in improving productivity of borrow-checking, but it still remains quite far behind GC.
Yikes. I could probably clean this up a little, but not a lot.
I've been working on this code for months and I still have no idea if my internal b-tree functions should be taking a &mut self, self: Pin<&mut Self> or a NonNull<...> or something else. The compiled code is blazing fast, and being able to control behaviour so clearly with a few parametric type parameters is amazing.
But the process of figuring out the best way to code it up is awful, and it requires all of my attention and capacity. I doubt Rust will ever gain the sort of mainstream usage that javascript & Go have because of how complicated otherwise simple problems can become. Its a great language for the linux kernel and web browsers. But I can't imagine many normal programmers will want to build regular websites and apps in rust.
GC languages are slower, but credit where its due - they make it so much easier to just dive in and spend all your braincells thinking about your problem domain.
I have been in plenty of hills since I wrote those first BASIC lines on a Timex 2068.
Just like consumers don't care if their favourite music application is written in Electron, as long it plays their favourite album on their newly acquired speakers, they also don't care what type system was used to deliver such experience.
With Go finally getting generics, it will be pretty alright, 99% of distributed computing applications need not care about zero GC code, only proper use of value types.
Your point about Go shining a lot more brightly when working with a team which has mixed skill levels was one of the design goals of Go, so it seems they’ve succeeded on that front if you’ve come to that conclusion independently!
IMO, Go is the easiest language to read by far. Python can be, but lets one use a lot of hard to figure out magic. Go is very much WYSIWYG, and to me that's its greatest strength. It's easy for anyone to jump in about anywhere. And that lends itself well to teamwork.
I find that's only true so long as the problem you're solving fits well into Go's view of the world.
If you want to make a data structure holding the equivalent of a parametric enum (or tagged union from C), go is very awkward to use compared with richer languages like Swift or Rust. Go is also awkward if you want to implement custom generic data structures.
Eg, this[1] code I wrote a couple years ago for doing text based operational transform became about 1.5x longer in Go compared to rust or typescript because its so awkward to express a parametric enum in go. And it was much harder to read & more buggy as a result. Sadly I lost the go version of the code. I'd be curious if someone with more go experience could do a better job, but I'm skeptical.
Hopefully the situation improves somewhat when generics land.
> I can imagine Go shining a lot more brightly when working with a team which has mixed skill levels.
Thank you for expressing that thought. However: I came quite to the opposite conclusion. I´m currently working in a small team with members that are not so versatile in coding, coming from a JavaScript background. It has been surprisingly easy to teach them Spring Boot. You follow a typical layout and everything falls into its place. I have my reservation that the same holds true for Go. It feels like there is much more room for confusion. Do I use global functions or receivers? How do I inject my dependencies and where do manage them? How do I build constructors and do I really need them? I found surprisingly little advice on those topics.
> but there is still a big productivity gap between the borrow checker and GC.
The borrow checker and GC aren't really doing the same thing though. One very important distinction in some fields (but unimportant in others) is that GC only really cares about memory resources. So you are (or should be) doing explicit manual cleanup for all non-memory resources in a GC language. Rust isn't, in Rust we don't write explicit resource cleanup for a programmable interrupt controller, or a file, or a database connection -- the resource knows how to manage itself, and Rust promises to tell it immediately when it falls out of use, whereas a GC can't promise to ever clean up which is why Go doesn't even bother providing a means to do this automatically for your non-memory resources when they're garbage collected (Java does, but again, no promises it ever fires, so, don't rely on this).
> Go doesn't even bother providing a means to do this automatically for your non-memory resources when they're garbage collected (Java does, but again, no promises it ever fires, so, don't rely on this).
It sounds like you are talking about finalizers? If so, they exist in Go in a similar way to the Java feature [1].
Finalizers aren’t quite the same. They run when memory pressure exceeds some threshold, but the GC analog for borrow-checking is something that will clean up a resource when the pressure on the resource in question exceeds some threshold (e.g., file handles). And even then I’m not sure if that ticks all requirements for resource management.
I meant to respond only to the quoted section comparing Java and Go, which states that Go doesn't have a way to do cleanup on GC (finalizers). I agree with you (and the two other replies and the quoted section itself) that finalizers are not a replacement for RAII etc.
I don't use them for resource cleanup, but I think they probably make sense for library authors as a back-up mechanism (for example, the standard library has a finalizer to close file handles, and the current runtime does GC every two minutes in the absence of memory pressure). You can also explicitly run the GC in Go, and while generally there's a lot of problems with that idea I could maybe imagine it being viable for certain unusual workloads.
You usually want to release non-memory resources deterministically, but finalizers in C#/Java/Go are not deterministic. That's why finalizers usually are not used for resource management. Instead, C# has using statements, Java try-with-resources, Go defer statements.
Huh, I also thought Go had no concept of finalizers. That is a bit... awkward way to set a finalizer. But thinking it again, it may just be due to my unfamiliarity.
That is a misconception, RAII like code can be achieved in two ways:
- defer (possibly add a go vet check for when missing it)
- when Go 1.18 brings generics, the withFunc pattern from FP where lambdas get given a resource released on function exist, thus having regions/arena like resource management
The description for go vet says, "it should be used as guidance only". Go vet is in fact a linter (go lint is also a linter, but one focused on style).
So what you're doing there is adding a control to try to mitigate the consequences of a language failing. Go doesn't actually prevent you from getting this wrong, but a linter such as go vet can flag cases where it suspects you screwed up, and maybe you'll catch the worst mistakes most of the time this way and by having usage conventions.
We shouldn't mistake this for equivalent capability. The complicated cases that tempt people to ignore or switch off such linting are exactly the cases most likely to have an undetected problem.
So this is "RAII like" only in the sense that "Just remember to do it properly" is RAII like, and we could make exactly the same claim for C.
Fair. However, there continues to be an unresolved tension in the C++ community about whether they want C++ to focus on its considerable legacy by ensuring enduring compatibility ("No ABI breaks, ever!") or to continue growing and changing even if that means not everybody can follow ("Performance trumps compatibility, ship it!"). Although we can anticipate some sort of compromise, it just isn't possible to have a C++ 2x that delivers everything the modernizers want yet still runs people's technically conforming C++ 11 code unchanged with that binary DLL they've got no source code for. Some people will be unhappy, maybe everybody in the C++ community will end up unhappy.
In the "no breaking changes" case I agree that C++ objects that live in the free store are in the same place as Go, the programmer has a responsible which they may not fulfil, to manage this resource and a linter can only help mitigate this problem.
But plenty of people including Stroustrup want to do lifetime management, despite potential breaking changes from that, and under lifetime management the compiler has visibility into your object lifetimes and can reject programs which inadvertently leak.
Now, that doesn't (can't) make leaks impossible, but it means any leak is now in some sense "on purpose" and would happen for GC'd resources too, it isn't just an accident. For example Rust's mem::forget will prevent the drop happening for the object you're forgetting, but it's not as though you type mem::forget() by mistake. You clearly wanted to achieve that (e.g. you stole the underlying Unix file descriptor from a File and sent it over a socket to a separate process, so now cleaning up that descriptor is the Wrong Thing™) and incorrect usage is not the same category of error as forgetting a with clause in Python.
defer (/using/with) all require the programmer to remember, and write, O(use site) times, the code to release the resource.
RAII destructors, on the other hand, do not permit the coder (at the use site of the type) to forget, as the destructor is invoked automatically when the variable goes out of scope.
In my time reviewing Python (also GC, "with" provides a similar functionality), this is a very common error.
Agreed, but per my original post, the additional safety that Rust affords protects against a very small share of bugs in SaaS applications. The value proposition isn’t good because you trade so much productivity for a small improvement in quality, and productivity is at a premium and quality (sadly) can be mitigated in other, less costly ways (e.g., progressively rolling out new changes, continuous deployment i.e., bugs can be patched in hours rather than months, etc).
I have the opposite view on the type systems: Rust's type system is really great for business logic because it's really expressive (based on algebraic data types), but the language being made for low level stuff makes some things a pain. If you do something like DDD, ADTs really shine, especially when combined with pattern matching. I do agree that when you can afford a GC, Rust can be annoying.
> I really like Rust, but in my mind it's still relegated to the periphery of tasks that must be super fast (like C/C++, to your point) and/or low-level. Rust has made impressive strides at reducing the toil involved in compile-time memory management, but there is still a big productivity gap between the borrow checker and GC.
Even if it weren't for the performance I would use Rust over Go due to the lack of generics alone, as well as the associated general philosophy of Go that boilerplate is good and one should repeat oneself as much as possible.
I really, really like type systems, but I’ve built enough software to know that the returns for most applications diminish after a Go-like type system is in place, and there are many other factors that are more important than type systems (and Rust gets a lot of these right too!) such as performance, productivity, tooling breadth and quality (especially build tooling and package management), ecosystem, deployment/distribution story, learning curve, readability, etc. Ultimately I’m of the opinion that as cool as type systems are, for most software they need to exist in service of productivity which means you need a basic type system that helps you but gets out of your way. Type systems can’t catch many classes of bugs that are increasingly prevalent in a world where our services are getting increasingly smaller (or rather, where more infrastructure and networking are being put in between the logical components of our application) and we’re often able to detect bug fixes before they impact a significant number of users (progressive deployments, automated rollbacks, continuous deployment, etc) but the discourse around type systems hasn’t caught up yet.
It has to do with that one sometimes has to write 80 lines of code in Go to to the same as 3 lines of Rust code due to having to repeat oneself all the time.
One especially is in the case where it's boilerplate code that does not rely on any real deep thought but writes itself, and with the possibility of bugs in it, most of all. Anyone can make a simple typo or forget something in the endless boilerplate.
Maintainers are bottlenecked by the rate we can (re)read. Boilerplate is the problem that makes us want powerful languages; if it didn’t matter we could have stayed with assembly.
I hate to be "that guy" too, but coming from somebody who really likes Rust and is using it more and more (also at $dayjob now) we must admit that Go tooling is one step ahead. CPU profiler, allocation and heap profiler, lock contention profiler. It all comes out of the box.
Yes you have cargo flamegraph for profiling locally and you now have pprof-rs to mimick Go's embedded pprof support. But allocation heap profiling is still something I struggle with.
I saw there was a pprof-rs PR with a heap profiler but there was some doubt as to whether it worked correctly; to get a feeling of how that approach would work but without having to fork pprof-rs I implemented the https://github.com/mkmik/heappy crate which I can use to produce memory allocation flamegraphs (using the same "go tool pprof" tooling!) in real code I run and figure out if it works in practice before pushing it upstream.
But stuff you give for granted like figuring out which structure accounts for most used memory, is very hard to achieve. The servo project uses an internal macro that help you trace the object sizes but it's hard to use outside the servo project.
The GC makes some things very easy, and it's not just about programmers not having to care about memory; it's also that the same reference tracing mechanism used to implement GC can be used to cheaply get profiling information.
There are many differences, but the main ones are that it has less overhead when profiling, more thorough analysis features (Heaptrack's GUI is relatively simple compared to it), and the next version will have scripting capabilities for analysis.
Hmm, took a look (it's called Bytehound now), it has no PKGBUILD nor a `cargo install` crate so I can't install it in systemwide or user PATH, and requires Yarn to download and build JS dependencies (likely hundreds or thousands).
I tried `cargo install --git https://github.com/koute/bytehound.git`, but that results in "error: multiple packages with binaries found: bytehound-cli, bytehound-gather, interrupt, linking, lz4-compress, simulation".
> But allocation heap profiling is still something I struggle with.
Switch to a dumb allocator and then profile mmap calls or page faults? That should get you large allocations at least. It's a pretty crude proxy. The other allocation profilers I'm aware of cause significant slowdowns.
I currently intercept calls to
malloc/calloc/realloc/... and capture stack traces. This way I know how much memory gets allocated for each allocation site. Since allocations usually go through constructor calls, the presence of a constructor in the stack trace can let you infer how many structures of a given type are been allocated. Knowing how big they are is more tricky since allocation for the whole struct and its parts doesn't have to happen entirely in the constructor (some structures like vectors and hashmaps can grow, some structures can collect data from other sources and then hold onto them, etc)
Furthermore to know how the live memory is broken down between object type and allocation sites, you also need to track freed memory. This is significantly more tricky to do efficiently. I currently take an allocation sample every N bytes being allocated and use a poisson process estimator to scale the total allocated bytes.
The only ways I know to account for in use memory is to track every single allocation or to add some extra space for every allocation where we record whether a block had been sampled and of yes, what was its corresponding allocation event.
Looks like I misunderstood, I was suggesting something more indirect than tracing calls to malloc. If you're already doing that then I'm not aware of any better low-overhead solutions. There are high-overhead options such as valgrind's dhat.
The missing feature I was comparing was the ability of Go to provide good estimated heap and allocation profiles using minimal overhead on a production workload.
Go isn't low level in the same way that Rust is. Both languages have their places, but Go seems to be more of a better Python/JS, whereas Rust is a better C++.
Go should be understood as a replacement for C and C++.
If you're writing a compiler, or some other project where extreme high performance is not necessary (e.g., if you're willing to be, say, a factor of 2 slower than C) then Go is a good choice.
Go's performance is excellent, and should be sufficient for all but the most demanding applications. It's not a crummy scripting language like Python/JS.
> Go should be understood as a replacement for C and C++.
I think Go is a good replacement for the things people used C and C++ for twenty years ago, but less of a replacement for the things people use C and C++ for today.
Back then, C/C++ was your default "write big server program that needs to go relatively fast" language, and Go is targeting that. But in the meantime, Java got fast enough and hardware got cheap enough that Python, Ruby, and JavaScript have also eaten into that domain.
Today, I see C and C++ used primarily for embedded work and games. I don't see Go as being a great fit for either of those.
Go is a replacement for a certain subset of C/C++ projects that are I/O heavy, e.g. web servers. However the GC precludes it from many uses of C/C++, e.g. you couldn't write an AAA game in it, or a kernel, or a toaster.
Python/Js are interpreted by default and not good system programming choices. Go is compiled. This comparison totally misses the mark. In fact Go was developed mostly as C++ but sometimes Python replacement at Google.
And yet Go seems to have not captured many C++ developers, but rather developers from languages like Python, Ruby, and JavaScript.
The Go team originally wanted to replace C++, especially inside Google, but they didn't succeed. According to googlers on Hacker News, very few projects inside Google actually use Go.
The reason Go attracted these kinds of developers is precisely that it isn't a systems programming language. Systems people want something like Rust instead.
Under wider definition of “system software” you can consider db tech (etcd, cockroachdb, digraph) and compute management software (kubernetes, nomad, docker) to be included into that. Which is where Go is doing just fine. Lower level stuff is harder but because of gc not lack of generics
It def convinced me. I wrote c++ for almost 10 year before switching to Go. Even after c++11 have come out it was still night and day. Anecdotally the team i left at google rewrote some of my stuff from c++ in Go that I didn’t get to at the time
"The Go team originally wanted to replace C++, especially inside Google"
Im glad you said that and I was not just imagining that myself. I remember following the Go language closely in its earlier days and it was often spoken about as a "systems language" but it actually seems to have ended up settling as a language to write servers for people who are sick of OOP but still like their imperative C style code.
In my opinion they should have pushed for D instead of Go. It’s leagues better as a systems language if you can tolerate GC or if you can’t you just don’t use the GC.
That's just one of the ways to think about it. Java, .NET (I felt like adding these two to expand on the comparison), Python and JS are all languages with a relatively high level of abstraction and large and useful ecosystems surrounding them. They're pretty popular for all sorts of application development, but all suffer from certain problems:
- Java has lots of brittle reflection in some libraries and JDK can be finicky, especially with GC tuning
- .NET needs a runtime, historically there's Mono, now there was .NET Core and now there will be just .NET, though in some cases there's also IL2CPP and so on
- Python not only generally runs slow, but also has problematic package management, especially with vent
- JS (Node in particular) has similar package management woes as well as really fast package deprecation
Go at least partially solves some of those problems, by being compiled, having decent performance, somewhat rich ecosystem and passable package management, all while the language remains usable.
Lots of applications and some tools will get written in Go because of this, because it's pretty reasonable to use in most cases.
In comparison, C, C++, Zig and Rust would all be better suited for systems level programming or embedded development - they typically let you write more performant and less memory hungry code, at the expense of foot guns and slower development.
I do both C# and Go so have no axe to grind here. Just two small points though.
> .NET needs a runtime
It doesn't. With a single command .NET Core can produce stand-alone single-file cross-platform deployables needing no SDK, Framework, runtime, or other dependency on the server.
> Go at least partially solves some of those problems, by being compiled, having decent performance, somewhat rich ecosystem and passable package management, all while the language remains usable.
It does, and I've been a big fan of Go for a fair few years now. However every point made in that sentence applies equally to .NET too.
The one area where Go beats C# (and most others) hands-down for me is the build time. It's a whole order of magnitude (possibly several) faster than most alternatives.
Partially agreed, many times it's just a patch over a codebase that's stuck only running on JDK 8 - people expect tuning to be sufficient when the code is slowly rotting.
I'd argue that most systems that won't die out eventually will need a rewrite, or alternatively, in the modern day we'll see more and more polyglotic systems popping up.
Therefore it always makes sense to explore the best options for a particular bit of development, regardless of whether it's Node, Python, Java, Go or something else.
But it wasn't just "interpreted languages". Really the only interpreted languages in "the niche" were Python and a little bit of Ruby, but there was also a whole bunch of Java, C#, and C++.
Ah, the desperate Rust promoter crew enters the Go thread ...
Rust is great for language enthusiats doing hobby projects or for learning. The Rust book is great. Rust has great language features.
Having that said, Rust is the right tool for the job for a very small niche. Basically, if you would have used C++ before and don't need much developer reach or mature libraries. And only for the rare cases you really cannot affort a GC (even though Go's GC is highly optimized).
Go on the other hand, is an industrial strength proven and mature general purpose language. It is the best fit for various kind of networking application. Especially APIs, but also infrastructure where you can live with an GC. CLIs are great with Go as well.
If you're working on a professional grade project (where the GC is acceptable), Go is much superior than Rust in all regards.
Rust is advertised for years and years and didn't have it's breakthrough yet. This empirical fact cannot be ignored. There're reasons for this, of course. Some are:
- Writing Rust consumes so much more mental power with so little gain. That mental energy should be directed to solving the problem.
- Go makes everything besides your problem at hand easy. You can focus on solving your problem, not fighting the Borrow Checker
- Rust's ecosystem is not reliable. Many essential libs are one-man-shows. Version 0.1 everywhere.
- Rust is for and by language enthusiats. If you need to rely on libs for longer than a couple of years it is a hight risk for your project
- In terms of real world performance: Go is so close to Rust that there're very very very few use cases that really need that marginal gain
Frankly, people are making boatloads of money off of software written in Python and JS--languages which are both far less safe than Go and far slower and yet Go is on par with those languages with respect to productivity (I argue it's more productive due to its static typing features, but others argue it's less productive presumably because of the learning curve). Most software doesn't need to be so fast or correct.
Also, I work in distributed systems and SaaS (software as a service). In this world, most errors are silly type errors ("undefined is not a function", forgetting to `await` some async function, etc) that would be caught with a flat-footed type system like Go's. Beyond that, the next largest bucket are issues that even a sophisticated type system like Rust's wouldn't help with, such as infrastructure issues (missing/misconfiguration of some cloud permission, networking rule, etc), misconfiguration of the service, race condition with many processes accessing the same networked resource, deployment error, etc. This means that Rust's stated advantages aren't really as nice as they initially sound.
As a programmer, I can appreciate a super fast language with a strict type system; however, as an engineer or a technologist, Rust is rarely a good fit for me. Go seems to be a lot closer to the sweet spot, which makes sense because it was developed expressly with distributed systems in mind.