[Foxboron]

>A Go project should only require the Go compiler to build successfully.

But they don't. The go compiler doesn't support yarn, npm, protobuf, open-api generators, doc generators like md2man, go-bindata-assetfs, gox and everything you need to complete the code generation done in modern Go applications.

So how do you orchestrate this? People use Makefiles, bash scripts, go scripts and everything in between and combined. It gives you a plethora of bewildering and confusing build options which can't be solved with `go build`, and neither with `make` in a straight forward fashion. Add `go get -u ... && go mod vendor` with some `npm install` in the Makefile, along with some overriding and/or ignoring `$GOFLAGS`, `$LDFLAGS` and `$CGO_LDFLAGS` and you got yourself an ecosystem hostile for packaging and compilation.

A go project can't use `go build` by itself - but it can't really use the Makefile either as people overengineer the process.

But let me stress this. Always use plain Makefiles over any other methods. It's there and has been used for decades for a reason.

[jerf]

I am generally able to keep things so that during development, "go build" works. This is great for quick turnarounds during dev and early testing.

Since I consider it necessary for production executables to explain where they came from, and I therefore embed the Git commit hash and other such information into the executable, it is simply a non-starter to ship my production executables coming from a bare "go build". So I have a shell script-based release process for all my projects that handles all that. In the spirit of Go, it's actually something I've been copy/pasting from project to project, because it always turns out that each one deviates so far from the "base" for its own individual reasons that there's hardly any reason to try to extract out any sort of "base" script. Since my Go projects are generally small-ish (I don't necessarily do "microservices" but I don't do massive monolithic exes, not because I'm super-awesome but just due to the domain I'm working in), make doesn't bring a whole lot of value since the entire final compile for me is under 5 seconds. YMMV. This script also handles tagging in a coherent way and some other basic software engineering maintenance tasks.

I really recommend this approach, and if necessary, doing the work necessary to maintain the ability to quickly do just a "go build". It helps "go test" keep working properly too since the rules for having "go build" work are pretty much the same as having "go test" work.

(In fact, as appropriate, I recommend it out of the context of Go, too. It just isn't always as easy. But prioritize keeping that dev turnaround down. If you're sitting there staring at a build process, use that time to think about how you can cut it down. It isn't just about the raw temporal efficiency... it's about your human brain and the way it stays motivated. The time loss of a one minute build is utterly insignificant next to the slowly-drained motivation and enjoyment the one minute build costs you.)

[Foxboron]

Let me stress that packaging software and deploying software has two different concerns. Your script might work wonders for deployments and shipping it to your infrastructure. But it might be completely broken if anyone want to package up the code and redistribute the software in a linux distribution.

[jerf]

Absolutely it would be. But I'd be in a lot of trouble if the software in question showed up in a Linux distro. :)

[alwillis]

I went through a phase of using Grunt, Gulp, npm scripts, etc. for web projects.

Recently, after reading this article on make[1], I decided to give it a try and I've been pleasantly surprised with how much easier using make has been vs. some of the FUD I've read about it.

That's not to say there aren't some issues, but that's true of something that's been around since the '70s, including Unix itself.

But that also means it can address almost any situation where files need to be generated and there are dependencies involved, including what I'm doing with generating a static website, compiling and minifying CSS files (or not minifying but including a sourcemap if it's a dev build) and deploying to a production or staging server.

I don't love the syntax but it’s not that bad once you get used to it. Actually, I prefer it to the seemingly endless nesting of JavaScript objects in a Grunt or Gulp configuration file. My next step is to take advantage of Vim’s built-in support for a make-based workflow.

[1]: https://www.olioapps.com/blog/the-lost-art-of-the-makefile/

[throwaway894345]

I agree with your point, but make solves the problem rather poorly. Besides the poor UX, it only knows the thing it built most recently, as opposed to maintaining a cache of things it has _ever_ built. And since it doesn't have a cache, it definitely doesn't have a distributed cache, although you could conceivably try to shoehorn NFS or something similar. Lastly, it's not reproducible. It's not going to fail a build if you accidentally let it depend on a file that isn't a formally specified dependency. These are all important concerns, especially for a CI environment.

Something like Bazel theoretically solves this problem, but it's poorly documented and non-trivial to use or operate. Other Bazel/Blaze derivatives are worse with respect to usage/operation/correctness. Nix improves on correctness but is even harder to use/operate. There's a lot of room for improvement in this space.

[Foxboron]

I agree with the observation. But the alternatives are grossly over-engineered or a case study in NIH. I'm sticking with Makefiles until someone can present something better.

>Something like Bazel theoretically solves this problem, but it's poorly documented and non-trivial to use or operate. Other Bazel/Blaze derivatives are worse with respect to usage/operation/correctness.

I have night terrors from listening to two of our packagers fighting against bazel, tensorflow and 10 hour compile times.

>Nix improves on correctness but is even harder to use/operate. There's a lot of room for improvement in this space.

I don't think Nix improves anything when you are stuck writing a weird javascript derivative. This comes from a packager writing bash for a living.

[shantly]

Maybe I'm unusual, but I avoid Java-based tools (like Bazel) because my Java environment often seems to be broken for any given piece of Java software, for one reason or another, and I don't want to add "make sure my Java environment is OK for all the Java tools I'm using" to the getting-started steps for any non-Java projects I'm on.

And yes, I mean the JVM, not the Java build tools. It's one of the reasons I go "ugh" when I realize I'm gonna have to run something written for the JVM. There's a decent chance I'll lose time configuring it to get it to work.

[throwaway894345]

> I agree with the observation. But the alternatives are grossly over-engineered or a case study in NIH. I'm sticking with Makefiles until someone can present something better.

I agree that they are complex beasts, but that complexity is incidental, not essential (some might argue "a lack of engineering" rather than "overengineered"). The documentation for these tools is also pretty atrocious on average. However, I don't think they're NIH insofar as their direct ancestor (Blaze) was never publicly available and Bazel didn't exist when those original Googler pilgrims brought their build-system ideas to Facebook, Twitter, Foursquare, etc. But nevertheless, there are half a dozen shitty tools instead of one decent tool. Worse, they're pretty much all designed for use in large organizations' monorepos--organizations who can employ people who are specialists in operating/maintaining these tools.

> I don't think Nix improves anything when you are stuck writing a weird javascript derivative. This comes from a packager writing bash for a living.

The improvements are certainly not uniformly distributed, nor are they sufficient to really justify its mainstream use, IMHO. :)

[rat9988]

>It's there and has been used for decades for a reason.

I have always had problems to comply with such statements when the reasons aren't given.

[baq]

either the parent edited his comment or you haven't read it, you're quoting the only sentence that hasn't got a reason in it.

[Sean1708]

None of those reasons explain why a Makefile is any better than a scripting language.

[coldtea]

Here's 3 reasons:

- Makefiles are a de-facto standard. People know them and those who don't can read how they work in 2000 tutorials. They shouldn't have to read your (and everybody's) custom (and different) script to build a project they've downloaded.

- Makefiles are specialised to the tasks of building, running tests, etc. A scripting language is general purpose. As such, it encourages adding all kinds of crap, from overcomplicated steps, to security issues.

- Makefile just needs make which is part of the core set for any distro and works fine on Mac and Windows (WSL or elsewhere) as well. Users shouldn't have to install a scripting language (or even a specific version of one) just to build a project.

[erik_seaberg]

Make is better than de facto; POSIX specifies it.

[rat9988]

Thank you. I had to reread the comment two times to make sure I didn't miss something. Glad to see I'm not the only one.