Hacker News new | ask | show | jobs
by jrockway 2385 days ago
If you download some random tar file with go code in it, I agree that you should expect to be able to build it with only go installed. "go get" depends on this, and largely works well!

But the day to day act of developing a system that uses go involves more than just building a go binary. Your go program might depend on things like generated protocol buffers. You need some way to regenerate those when you edit the definitions. That then involves having the right version of protoc installed and also having the right version of protoc-gen-go. The go compiler can't help you there. go generate suffers from the same problem; it's not automatic, so you can pass it the wrong dependencies (generation tool flags, version of the generator, etc.).

People are using makefiles as a convenient place to write all these extra instructions. "What flags to I pass to protoc?" "What flags do I pass to docker build?" Why document it when you can "make protos" or "make container"?

Unfortunately, make isn't actually good at this. It doesn't version the generation tools (or itself), so you will end up with vastly different results on different machines. The result is things like a 300 line diff to a generated protobuffer because the second engineer to work on that file happened to have protoc 0.7.7 instead of protoc 0.6.42, or they installed protoc-gen-go@master instead of protoc-gen-go@v1.2.3. Make doesn't care. It exited with exit status 0, so it must have worked.

What started as a nice way to write down some instructions for hacking the code has now become a giant mess. Reasonable makefiles can only ever work for one person on one computer at one point in time. At that point, they might as well be a README.md. At least the README can mention the version numbers of the dependencies, and, most importantly, can wish the reader luck.

There are two long-term solutions. One is to only use go. Write a program that reads the protos at runtime. Write a program that runs your Typescript through a hand-written compiler whose source code lives in your project at runtime. Now you only need "go build". This is... impractical, though. It's a nice ideal, but you'll never get anything done in the real world.

So what you really need is a real build system that captures every dependency, knows about high level tasks ("make foo.proto available to a go program"), and knows every dependency between files like the go compiler does. With such a tool, you can get a working build on every computer with no instructions or manual setup. And since it is carefully written to understand what it's doing, you can get reliable incremental builds. (The full build and your incremental build should have the exact same md5sum of the resulting binary.)

Such a tool does not exist. bazel is close. If you have to build more than just go files, you probably want to look into it. It's crazy. It's a lot of work. Don't do it if you're the only developer on the project. But if you want 10 random people to be able to build a project that's written in more than one language, you have to invest in some sort of tooling. Make is good for a one person team. Make can be scaled to do crazy things poorly (hi, Buildroot!). But it's probably not what you want to be using. If a README isn't good enough, you need a real build system.