[Night_Thastus]

I am extremely interested in this.

I am stuck in an environment with CMake, GCC and Unix Make (no clang, no ninja) and getting detailed information about WHY the build is taking so long is nearly impossible.

It's also a bit of a messy build with steps like copying a bunch of files from the source into the build folder. Multiple languages (C, C++, Fortran, Python), custom cmake steps, etc.

If this tool can handle that kind of mess, I'll be very interested to see what I can learn.

[unddoch]

I wrote a little GCC plugin for compile time tracing/profiling, if that's something you're interested in: https://github.com/royjacobson/externis

[Night_Thastus]

I just went ahead and tried it out :)

I can get it to work for some sub-sets of our project, but for quite a bit of it I get the following error:

cc1: error: cannot load plugin /opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/plugin/externis.so: /opt/rh/gcc-toolset-13/root/usr/lib/gcc/x86_64-redhat-linux/13/plugin/externis.so: undefined symbol: _Z14decl_as_stringP9tree_nodei

I suspect this is because these are C or Fortran sub-projects. I'm looking for some clean way to tell Cmake to apply externis to all the C++ only subprojects if possible. I'll see what I can come up with.

I'd also like to know, if multiple GCC commands end up pointing to the same trace.json, especially in a parallel build, will externis automagically ensure that it doesn't step over itself?

[Night_Thastus]

I figured out a way to set it at a top level, so it only happens with C++ files:

target_compile_options(${NAME} PUBLIC

$<$<COMPILE_LANGUAGE:CXX>:-fplugin=externis -fplugin-arg-externis-trace-dir=(where I want to put traces)>

)

But as I suspected, it is not a single trace file. It's thousands of trace files. Is there some way to collate all the data into one larger picture of how the build progressed?

[hagendaasalpine]

Tsoding wrote https://github.com/tsoding/nob.h, single header C library for cross platform builds, only requirement is cc. GDB profiling tools can then be used to look at your build step. It's a neat idea. I suspect this is not an option but Nix is great build tool if you are dealing with multiple languages.

[jppittma]

Btw, he has a YouTube channel and streams. I recommend it if you’re seeking imposter syndrome.

[wakawaka28]

It's not "nearly impossible" but actually built in: https://cmake.org/cmake/help/latest/manual/cmake.1.html#cmdo... For the actual compile time you can easily insert a wrapper script. To be honest I haven't done that in over 4 years, but it has been done by many and it is easy.

There may be times when CMake itself is the bottleneck but it is almost certainly an issue with your dependencies and so on. CMake has many features to assist you in speeding up your compile and link time too. But it would take a series of blog posts to describe how you should try to speed it up.

[Night_Thastus]

Does that just profile cmake's time to configure and generate, or the underlying compile of each file as well? Configuration and generation are only seconds when done from scratch - the build is more like 20 minutes.

Just trying to add that argument with 3.26.5 on Rocky Linux 9, I get 'Unknown argument --profiling-format=google-trace'.

Not sure why, as cmake --help clearly states it should be there...

  --profiling-format=<fmt>     = Output data for profiling CMake scripts.
                                 Supported formats: google-trace
  --profiling-output=<file>    = Select an output path for the profiling data
                                 enabled through --profiling-format.

[tom_]

You might need to specify --profiling-output= as well. I get an error fdrom cmake 3.31 if only the format is provided: CMake Error: --profiling-format specified but no --profiling-output!

Anyway, it looks like it only profiles the configure/generate steps. Not much use on Linux, but on Windows/macOS, perhaps. Due to lack of any standard package manager, it's a good idea to build every dependency from source on those OSs, and the time can mounts up.

My project is not that large, but it takes 1 minute to configure from scratch on Windows, and 10 minutes (!) on macOS.

[Night_Thastus]

I specified both. In any case, yeah, cmake time isn't useful in my case.

Bizarre that you see 10 minutes on MacOS. Something's definitely busted there. It's not even that bad for me on Windows, and that's saying something.

[tom_]

For the record: the cmake profiling data did help a bit. On macOS it seems that the configure stage does 200+ try_compiles, largely due to SDL2, and each one somehow takes ~2.3 seconds. So that's 460 seconds right there. And regarding the total time, it's actually more like 480 seconds. (I must have misremembered! Or perhaps my laptop is measurably faster when its integrated GPU isn't driving 2 external displays.)

On Windows: 50 try_compiles, about half and half SDL2 and libuv, and each one takes ~1 second.

I don't think either of these try_compile turnaround times is acceptable (what is it doing?! I bet it's like 50 ms on Linux) but the total figure does now feel a bit less mysterious.

[wakawaka28]

You might also like my suggestions in this comment: https://news.ycombinator.com/item?id=44931551

[wakawaka28]

That does just do CMake time. Sorry if my comment was confusing. I forgot that I used a wrapper or something myself to collect the compile times. Anyway, my point is, you aren't the first person to have this issue. I've definitely profiled compile times before and viewed a flame graph with some gadget made by Google. The easiest way you to do it may depend on your compiler and build system. You could attack the problem from Ninja or Make, or by substituting your compiler binary for a wrapper script.

I also want to point out that if you want Ninja, it is a very minimal binary to build with no dependencies that wouldn't already be installed.

This is one of the simpler ways to find out what's going on: https://alangrow.com/blog/profiling-every-command-in-a-makef... Instead of replacing the shell you can also replace CC or CXX variables (if I recall properly) either in CMake or the Makefile to replace your compiler with a wrapper script that logs stuff.

More generalized Makefile profiling: https://github.com/konturio/make-profiler

Profiling with Clang: https://aras-p.info/blog/2019/01/16/time-trace-timeline-flam... (Maybe what I used last time...)

I am pretty sure you can get shiny outputs from a free profiling tool, but I think you can search for that yourself. It should be easy to find something. Good luck!

[phaedrus]

When I was trying to improve compile time for my game engine, I ended up using compiled size as a proxy measure. Although it is an imperfect correlation, the fact that compiled size is deterministic across build runs and even across builds on different machines makes it easier to work with than wall clock time.

[mlsu]

Wait, this is not intuitive at all for me.

If the compiler is working harder wouldn't that result in a more compact binary? Maybe I'm thinking too much from an embedded software POV.

I suppose the compiler does eventually do IO but IO isn't really the constraint most of the time right?

[staticfloat]

While you can cause the compiler to run longer to squeeze the binary size down, the compiler has a baseline number of compiler passes that it runs over the IR of the program being compiled. These compiler passes generally take time proportional to the input IR length, so a larger program takes longer to compile. Most compiler passes aren't throwing away huge amounts of instructions (dead code elimination being a notable exception, but the analysis to figure out which pieces of dead code can be eliminated still is operating on the input IR). So it's not a perfect proxy, but in general, if the output of your compiler is 2MB of code, it probably took longer to process all the input and spit out that 2MB than if the output of your compiler was 200KB.

[johannes1234321]

Of course there are the cases where a huge template structure with complex instantiation and constexpr code compiles down to a single constant, but for most parts of the code I would assume there is a proportion from code size, via compile time to binary size.

[fransje26]

> I am stuck in an environment with CMake, GCC and Unix Make (no clang, no ninja) and getting detailed information about WHY the build is taking so long is nearly impossible.

I have a similar problem, with a tangential question that I think about from time to time without really having the time to investigate it further, unfortunately.

I notice sometimes that CMake recompiles files that shouldn't have been affected by the code changes made previously. Like recompiling some independent objects after only slight changes to a .cpp file without any interface changes.

So I often wonder if CMake is not making some file more inter-dependent than what they are, leading to longer compilation times.

[1718627440]

Can you set CC=time gcc ?

[pklausler]

strace might help, if you have it.

[touisteur]

perf record -e "syscalls:,sched:" is my go-to there - or just apply it on whole-system, aasone does.