[jrockway]

The problem with Make is that it doesn't know what it's building, so it can't do anything smart. For incremental builds to work at all, you have to supply the smarts. Having worked on a number of make-based projects (I am most scarred by buildroot), I can tell you that people make mistakes with build rules. The project then devolves to doing a clean build for every change, turning what could be a few milliseconds of CPU time into a 30 minute rebuild.

The idea of these build systems like Bazel is that the rules are correct so that you don't have to worry about writing correct rules, and you have a high probability of an incremental build producing a binary that's bit-for-bit identical to one from a full build. The result is that you don't do full builds anymore, and save the latency of waiting for things to build. (That latency shows up in the edit-build-test cycle, how long it takes to deploy software to production to fix an emergency bug, etc. So it's important!)

[dblotsky]

I am personally not convinced that any build system can be correct and general, but perhaps that’s my lack of experience speaking.

On that 30 minute note though: so, how big does the project need to be in order for Make not to be enough? And at that size, why wouldn’t the project invest the extra week it takes to get the Makefile correct?

[laurentlb]

It's easier to adopt a correct build system when your codebase is still simple. It can save you time, because it can prevent lots of unintentional errors over time (e.g. you might not notice when you get an incorrect result after an incremental build).

In my opinion, this is a bit similar to languages with static vs dynamic typing.

If your current setup works well for you, it makes sense to keep it, though.

[jrockway]

I completely agree with you. To give the build system the understanding it needs, you have to give up the generality.

For me, I've found that the results are best when you use a build system and you use it exactly the way the author intends. For example, Go's built-in build system is so good that you don't even notice it's there. It automatically updates its configuration as you write code. It does a perfect incremental build every time, including tests. The developer experience is basically perfect, because it has a complete understanding of the mapping between source files, dependencies, and outputs. But, it is not extendable, so you're screwed when your Go project also needs to webpack some frontend code, or you need to generate protocol buffers (which involves a C++ binary that runs a Go binary), etc. So, people bolt those features on, and the build system becomes more general, but not quite as good. (Then there's make, which is as good or as bad as you want it to be.)

I think super small projects often do get their Makefiles right. But you can manually build small projects with something like "gcc foo.c bar.c -o foobar -lbaz", and so you don't really benefit from any improvement over the bare essentials. (Nothing wrong with keeping your projects tiny in scope, of course!)

But, sometimes you don't have the luxury of a super small project, and the Makefiles become quickly unfixable. Like I said, I am most scarred by a buildroot project I worked on (that's embedded Linux, basically). It never built what I expected it to build, and to test anything reliably I either had to yolo my own incremental build or wait a while for a full build. My productivity was minimal. I could switch between client-side and server-side tasks on that project, and so I really only touched the client if it was absolutely necessary. I would never be productive enough to undertake a major project that truly added value with that kind of build system, so I let others that didn't have the server-side experience write the client-side stuff. In that case, the poor build system silently cost our team productivity in terms of artificially splitting the team between people who could tolerate a shitty developer experience and those who couldn't.

I don't think anyone has fixed the buildroot problem, either. If you want to build a Linux image today, you are stuck with these problems. Nothing else is general enough to build Linux and the associated random C binaries that you're going to want to run.

[dblotsky]

Yeah, that tradeoff between generality and correctness really seems tyrannical.

It kind of feels like the best trajectory is to start small with a general build system, and upgrade as needed? And then if you are confident the project will grow, starting with the specific build system fine too.