[thechao]

I've been burned by so many build tools over the years. I've finally settled (for C/++/asm) on the combination of Make + ccache: I build a _very_ paranoid Makefile that recompiles everything if it feels like anything changes. For instance, every rule that compiles a C/++ file is invoked if _any_ header/inc/template file changes. I let ccache do the precise timestamp/check-sum based analysis. The result is that (for large builds < 10MMLOC) I rarely wait for more than a few hundred milliseconds on incremental, _and_ I have confidence that I never miscompile.

I just wish that I had a high-performance replacement for linking that was cross-platform (deterministic mode for ar), and for non-C/++ flows. Writing a deterministic ar is about 20 lines of C-code, but then I have to bake that into the tool in awkward ways. For generalized flows, I've looked at fabricate.py as a ccache replacement, but the overhead of spinning up the Python VM always nukes performance.

[beagle3]

> I build a _very_ paranoid Makefile that recompiles everything if it feels like anything changes.

Do you have some kind of way to verify that your makefile dependencies conform to your source dependencies? Is clang/gcc tracking sufficient for your use case? What about upgrading the compiler itself, does your makefile depend on that? If so, how?

Have you considered tup[0]? Or djb-redo[1]? Both seem infinitely better than Make if you are paranoid. tup even claims to work on Windows, although I have no idea how they do that (or what the slowdown is like). Personally, I'm in the old Unix camp of many-small-executables, non of which goes over 1M statically linked (modern "small"), so it's rarely more than 3 secs to rebuild an executable from scratch.

> (deterministic mode for ar)

Why do you care about ar determinism? Shouldn't it be ld determinism you are worried about?

[0] http://gittup.org/tup/

[1] https://github.com/apenwarr/redo

[thechao]

> Do you have some kind of way to verify that your makefile dependencies conform to your source dependencies?

Nope. I explicitly use a conservative approximation—this guarantees correctness, over speed. Building everything every time with a clean tree is where I begin; I start optimizing after that.

> Is clang/gcc tracking sufficient for your use case? What about upgrading the compiler itself, does your makefile depend on that? If so, how?

Self-rewriting Makefiles (to consume the .d files), combined with the cleaning necessary for them, become a large technical debt—especially given the complexity of the Makefile needed to generate them. Modern CCen just aren't capable of this. Perhap Doug Gregor's module system will land in C21/C++21, and we'll see some good, then.

> Have you considered tup[0]? Or djb-redo[1]?

Yes. They are both don't provide significantly better correctness guarantees combined with sufficiently better performance to justify the cost to porting to older Unixen. (This is a consensus opinion at my shop; I, personally, enjoy tup.)

> Why do you care about ar determinism? Shouldn't it be ld determinism you are worried about?

Determinism let's me cache *.o/a/so/dylib/exe/whatnot without getting false-positives due to time-stamp changes and owner/group permissions in the obj/ar files (see ar(1)). ld is deterministic under all the CCen I use by setting the moral-equivalent of -frandom-seed.

[beagle3]

Thanks.

> this guarantees correctness, over speed.

Wouldn't "promotes" be a better word? what guarantee do you have?

> Self-rewriting Makefiles (to consume the .d files), combined with the cleaning necessary for them, become a large technical debt—especially given the complexity of the Makefile needed to generate them. Modern CCen just aren't capable of this.

Haven't needed it in a long time, but back when I did generating one for me was all of running the compiler with "-MD" in the compile phase, and including it in the Makefile - no special "make depend" phase, no noticeable slowdown. What technical debt are you ref

> Yes. They are both don't provide significantly better correctness guarantees combined with sufficiently better performance to justify the cost to porting to older Unixen.

Interesting. It is my experience that redo (from apenwarr) is trivial to run and use anywhere there's Python and isn't Windows -- it's almost as fast as Make, and it makes correctness guarantees that Make cannot (e.g., .o file replacement is atomic).

[thechao]

Maybe... 'prefer'. I'm more confident that a really conservative Makefile will build my code correctly.

My issue with -MD was not that it didn't provide precise (and correct!) dependencies; my issue was that the build system's most mysterious breakages are when modules (and dependencies) are changed. In that case, there are three situations:

1. Your .d files are out-of-date, and thus your build is broken;

2. You have to have a policy of "updating the .d files"; or,

3. Your makefile has to be .d savvy.

The last option is the one I see most often taken, but with rare success.

> it makes correctness guarantees that Make cannot (e.g., .o file replacement is atomic).

I wish Make wasn't so entrenched.