Hacker News new | ask | show | jobs
by dibanez 3461 days ago
As others have mentioned, I've resorted to brute force (parallel compile using 16 physical cores, good SSD, 32GB RAM). I have a 20KLOC library that I wrote myself (very few templates), that takes 20 sec to compile in parallel, meaning in serial it would take about 5 min. Then there is a giant 3MLOC system I work with (much templating), that takes 20 min to compile in parallel (consuming 15GB RAM), meaning in serial it would take 6 hours. One can also see in this 3MLOC system that it is the last few small files that take the longest to compile, because they include much of the rest of the system as template code in headers.
2 comments
thechao 3461 days ago
Preprocess your source 'cat' the results together; then, hand off to the compiler. On a 2015 MacBookPro I've seen upwards of 100kloc/s. Probably average 30--60kloc/s. With ccache my 5mloc code base takes as little as 300ms to do incremental compile & relink.

Another thing is to rely on the preprocessor for code generation; then you're dealing with memory-speed elaboration of the AST, rather than reading off of disk.

link
dibanez 3461 days ago
I did the single file trick for the small library that I control, as a configure option. It definitely does help, 2X speedup in serial compile time typically. however, you lose the ability to compile in parallel, which with my hardware gives a 16X speedup, so for now parallel compiles are the winner.
link
Joky 3461 days ago
Something smells with the structure of the project if a small change in a single file triggers a long rebuild. Including headers that define template should be "free", it is costly when you start instantiating all of them. There might be opportunity to restructure some part. Also precompiled-header or better: modules can help (not much with template instantiation though).
link
dibanez 3461 days ago
In my parent comment are listed compile times from scratch, although there probably exist one or two files that would trigger similarly long rebuilds (included indirectly by most other files). If you change a random line of code, chances are the rebuild is quite limited. As others commented above, this actually scares people away from improving the core files.
link