[CJefferson]

Saying this is an 'incremental makefile' isn't really correct, as changes to header files aren't going to lead to rebuilding.

So, either you need to manually keep references to which .h files you include in your Makefiles up to date, or start worrying about M / MM / MG / MP, and of course you'd like those to be re-run when you change your files, and suddenly your Makefile is an awful lot less simple.

This is the main reason I stopped teaching Makefiles in intro to C courses -- students often lost entire afternoons to forgetting this stuff, and failing to realise their mistake. It really shouldn't be any person's job to keep this up to date, when the computer knows it.

[anyfoo]

It is correct to call it an incremental Makefile, it merely doesn’t name the dependencies for you. A Makefile has exactly the dependencies you specify. It’s that simple.

That is not necessarily the most useful. Especially as your project gets larger, you probably will not want to specify every header file as a manual dependency, or alternatively rebuild everything every time you change a header file. (Though for me personally, either approach often works for surprisingly long.)

Then you can do things like letting the C preprocessor create the dependencies that get included in your Makefile (yes, common C compilers can create Makefile fragments). Or do something more complicated. Or switch to another build system, if your project really outgrew, or really mismatches, make.

But at its core, Makefiles follow a simple concept, are entirely language agnostic, and even with just the simpler concepts they remain useful for a long time.

[CJefferson]

Having 'incrementality', when it isn't actually going to incrementally rebuild things when files change, is (in my experience), worse that having no incrementality at all. Having to remember when I have to manually disable to incrementality is an annoying overhead, and easy to forget.

If you can remember exactly when you need to manually skip the incremental build, that's great for you, but I find Make has enough of these kinds of footguns I don't recommend it to people any more.

[anyfoo]

Meh, it can practically never be perfect, without diminishing returns that at some point invalidate the advantage of any incremental builds anyway.

As I said, you can let the C compiler generate your header dependencies for you. But that's not enough, because what's with header and libraries that are external for your project? Keep in mind that they will have dependencies, too, and at some point your transitive closure will be really big.

To a lesser degree, what's with a compiler, linker, or any random other part of your toolchain that changes (at least those are supposed to mostly keep a stable interface, which isn't the case at all for external libraries, but in some cases the toolchain can cause the footguns you mentioned).

A lot of the times, you don't need to rebuild everything just because a system header or the toolchain changed, but your build system would have a hard time knowing that. Because at some point, the halting problem even gets in your way (i.e. you can't really reliably detect if a given change in a header file would need rebuilding of a source file, which you would need for "fully incremental" builds). So it's always a trade-off.

Personally, I've fared very well with generated header dependencies (sometimes manually declared ones or none at all for small projects), and many other projects have, too.

YMMV of course, but I don't observe this to be bad. Most people who program C and use make are, I think, aware of what header mismatches can cause, and how to avoid that.

[CJefferson]

I think one fundamental difference between the two of us (I'm guessing here, let me know if I'm wrong), is I'm willing to cope with longer compile times, as long as I always get a correctly built executable (so there are no 'halting problems', if in doubt, rebuild it!), while you would prefer faster build times, even if that then requires sometimes having to do a little manual work when the build system doesn't realise it needs to rebuild? Of course, no system will be 100% perfect, but you can choose where your trade-off point is.

Two reasonable viewpoints, but probably effects how we do our building setups!

[anyfoo]

I think that's fair. On top of that, being a low level system programmer, I think I'm pretty good at realizing/intuiting when something critical changed that needs me to nuke the build dir, i.e. the manual work you mention to force a full recompile (and also good at noticing mismatched headers even through very weird symptoms).

I do have to switch between multiple SDK versions very often for example, and I always either nuke the build dir after doing that, or have separate build dirs per SDK in the first place.

[fuhsnn]

Yes, this is quite important for C projects as it leads to subtle bugs. I also long for a way to let make detect a change of $CC (new build of the compiler itself) in the context of debugging a compiler against third party makefile projects.

[anyfoo]

Have a step that takes the SHA256 or similar of $CC, make that a dependency. You’d need slightly more advanced make features if you want to make it “pretty” (and not just, say, touch a file through some shell code comparison), but this is a slightly more advanced request.

Or potentially just specify the path to $CC itself as dependency? Presumably, if the C compiler changes, the timestamp of its executable does too. (Bad if it goes backwards, though.)

[yjftsjthsd-h]

> Have a step that takes the SHA256 or similar of $CC, make that a dependency.

Careful, one more step and we'll start recreating nix in Makefiles;)

(Wait, could we? Should we?)

[CJefferson]

I often use a Python program called 'fabricate', which basically checks every file a command opens, and re-runs the command if any touched file changes.

I love it, and I hoped this would become the future of how build-systems are built -- you could automatically parallelise, rebuild, make clean, everything, just by tracking which files are read, and written.

It does add a bit of overhead, but I'd be willing to pay that for simplicity. Unfortunately, it doesn't seem to have caught on.

[anyfoo]

That is neat, but also super brittle, and full of abstraction leakiness, potentially leading to both false positives and negatives. Sounds like it would break down with ccache or distcc immediately, for example.

[CJefferson]

I never had any false negatives, they are basically impossible in this framework, except it does assume if a program, and all it's inputs are unchanged, the output is unchanged, so this is no good for programs which randomly produce different outputs.

ccache can cause things to get built twice, because the first run it sees it tries to read a file, then later writes to it, so it knows something different might happen if you run again -- but then again that is just ccache doing it's thing, so it's quick. Yes, distcc wouldn't work, but then again I find distcc super-brittle (so hard to make sure everything has the same compiler versions, etc), I may have used it 20 years ago, but I don't think I ever would now!

[anyfoo]

Fair!

[dezgeg]

The 'tup' build system also works this way.

[exe34]

You could have taught them `make clean` whenever there's any issues. I work with software engineers and I still have to remind them to try rm -rf ./build/* now and again, which always seems to solve the problem.

[CJefferson]

I do, but it’s easy for experts to forget, and when you are in your first few weeks of C it’s even easier to forget with everything else you have to remember.