[boris]

Could you summarize how you handle this in redo? Also what about the case where a header file does exist but is out-of-date and because of that triggers an error (e.g., version compatibility check with #error) -- how do you handle that?

[JdeBP]

I, for one, handle it with a tool that mimics the compiler's preprocessing phase and emits both redo-ifchange information for all of the headers that are used, and redo-ifcreate information for all of the non-existent headers that are looked for during the process.

    JdeBP %cat test.cpp
    #include <cstddef>
    void f() {}
    JdeBP %/package/prog/cc/command/cpp test.cpp --iapplication . --icompiler-high /usr/local/lib/gcc5/include/c++ --icompiler-low /usr/local/lib/gcc5/include/c++/x86_64-portbld-freebsd10.3 --iplatform /usr/local/include --iplatform /usr/include  -MD -MF /dev/stderr 2>&1 > /dev/null|fgrep redo
    redo-ifcreate ./cstddef ./bits/c++config.h /usr/local/lib/gcc5/include/c++/bits/c++config.h /usr/local/include/bits/c++config.h /usr/include/bits/c++config.h ./bits/os_defines.h /usr/local/lib/gcc5/include/c++/bits/os_defines.h /usr/local/include/bits/os_defines.h /usr/include/bits/os_defines.h ./bits/cpu_defines.h /usr/local/lib/gcc5/include/c++/bits/cpu_defines.h /usr/local/include/bits/cpu_defines.h /usr/include/bits/cpu_defines.h ./stddef.h /usr/local/lib/gcc5/include/c++/stddef.h /usr/local/include/stddef.h ./sys/cdefs.h /usr/local/lib/gcc5/include/c++/sys/cdefs.h /usr/local/include/sys/cdefs.h ./sys/_null.h /usr/local/lib/gcc5/include/c++/sys/_null.h /usr/local/include/sys/_null.h ./sys/_types.h /usr/local/lib/gcc5/include/c++/sys/_types.h /usr/local/include/sys/_types.h ./machine/_types.h /usr/local/lib/gcc5/include/c++/machine/_types.h /usr/local/include/machine/_types.h ./x86/_types.h /usr/local/lib/gcc5/include/c++/x86/_types.h /usr/local/include/x86/_types.h
    redo-ifchange /usr/local/lib/gcc5/include/c++/cstddef /usr/local/lib/gcc5/include/c++/x86_64-portbld-freebsd10.3/bits/c++config.h /usr/local/lib/gcc5/include/c++/x86_64-portbld-freebsd10.3/bits/os_defines.h /usr/local/lib/gcc5/include/c++/x86_64-portbld-freebsd10.3/bits/cpu_defines.h /usr/include/stddef.h /usr/include/sys/cdefs.h /usr/include/sys/_null.h /usr/include/sys/_types.h /usr/include/machine/_types.h /usr/include/x86/_types.h
    JdeBP %/package/prog/cc/command/cpp test.cpp --iapplication . --icompiler-high /usr/local/lib/gcc5/include/c++ --icompiler-low /usr/local/lib/gcc5/include/c++/x86_64-portbld-freebsd10.3 --iplatform /usr/local/include --iplatform /usr/include  -MMD -MF /dev/stderr 2>&1 > /dev/null|fgrep redo
    redo-ifcreate ./cstddef ./bits/c++config.h ./bits/os_defines.h ./bits/cpu_defines.h ./stddef.h ./sys/cdefs.h ./sys/_null.h ./sys/_types.h ./machine/_types.h ./x86/_types.h
    redo-ifchange
    JdeBP %

I also have a wrapper that takes arguments in the forms that one would invoke g++ -E and clang++ -E, tries to works out all of the platform and compiler include paths, and invokes this tool with them.

It's then a simple matter of invoking these redo-ifchange and redo-ifcreate commands from within the redo script that is invoking the compiler.

You can see this plumbed into redo in a real system in the source archives for the nosh toolset and djbwares.

* http://jdebp.eu./FGA/introduction-to-redo.html#CompilerDefic...

* https://news.ycombinator.com/item?id=15044438

[erlehmann_]

I use strace(1) look for stat(2) syscalls that fail with ENOENT. An advantage of this approach is that I do not have to imitate the C preprocessor, so parser differentials can never happen. The following default.o.do file from my blog post [1] handles the case:

  #!/bin/sh
  redo-ifchange $2.c
  strace -e stat,stat64,fstat,fstat64,lstat,lstat64 -f 2>&1 >/dev/null\
   gcc $2.c -o $3 -MD -MF $2.deps\
   |grep '1 ENOENT'\
   |grep '\.h'\
   |cut -d'"' -f2 2>/dev/null\
   >$2.deps_ne
  
  read d <$2.deps
  redo-ifchange ${d#*:}
  
  while read -r d_ne; do
   redo-ifcreate $d_ne
  done <$2.deps_ne
  
  chmod a+x $3

This approach is also used for building Liberation Circuit if strace is installed [2].

I think the compiler should output the necessary information. To quote Jonathan de Boyne Pollard [3]:

> As noted earlier, no C or C++ compiler currently generates any redo-ifcreate dependency information, only the redo-ifchange dependency information. This is a deficiency of the compilers rather than a deficiency of redo, though. That the introduction of a new higher-precedence header earlier on the include path will affect recompilation is a fact that almost all C/C++ build systems fail to account for.

> I have written, but not yet released, a C++ tool that is capable of generating both redo-ifchange information for included files and redo-ifcreate information for the places where included files were searched for but didn't exist, and thus where adding new (different) included files would change the output.

JdeBP, could you please release your tool under a free software license? I suspect it has fewer errors than the similar CMake approach [4].

[1] http://news.dieweltistgarnichtso.net/posts/redo-gcc-automati...

[2] https://github.com/linleyh/liberation-circuit/blob/master/sr...

[3] http://jdebp.eu./FGA/introduction-to-redo.html#CompilerDefic...

[4] https://github.com/Kitware/CMake/blob/master/Source/cmDepend...

[JdeBP]

Just for the record: My personal preference is for Clang and GCC to be instrumented to emit the names of both found and non-existent header files.