[xtrapolate]

> "The first fix was to avoid calling CommandLineToArgvW by manually parsing the command-line."

> "The second fix was to delay load shell32.dll."

If your build pipeline is continuously spawning processes all-over, to the point "delay loading" makes a significant difference - it's time to start re-evaluating the entire pipeline and the practices employed.

[wtallis]

Do you know of a build system that can handle a source tree as large as an entire web browser without spawning a lot of processes?

It's hard to tell what, if anything, you are recommending here. Pass thousands of files to a single compiler invocation? Ignore the problems and stop trying to make process creation and clean-up faster?

[im3w1l]

> Pass thousands of files to a single compiler invocation?

Sure. Or pass it a file with all the filenames. Or have the compiler work as a server that takes compilation requests over a socket. It's not like passing thousands of filenames between two processes is a deep unsolved problem.

[gpderetta]

Or just spawn thousands of processes which has been done for the last 40 years without particular issues.

[mannykannot]

So, the solution to concurrency problems is to serialize everything?

[fit2rule]

"Concurrency is everything serialised, properly."

[mannykannot]

The posts we are replying to here seem to have a very narrow concept of what 'properly' entails in this case.

[fuzzy2]

The second paragraph mentions that this is about a test suite. It has to “spawn processes all-over” to do its job.

[nwellnhof]

Technically, it doesn't has to. You can put your whole test suite into a single executable, making it run extremely fast for C/C++ projects where process startup is often much slower than running a single test unit. This approach is used in some OSS projects I've worked on but it also has its downsides.

[fuzzy2]

Some tests (e.g. unit tests) can run like this, yes. Other tests, including some benchmarks, are not meaningful when wrapped in a single process. Invocation speed and specifics matter.

[kornish]

Did you read the entire article? The whole point was that "delay loading" a particular DLL prevents a static analysis in the compiler from inserting hooks to perform expensive operations.

[xtrapolate]

> "Did you read the entire article? The whole point was that "delay loading" a particular DLL prevents a static analysis in the compiler from inserting hooks to perform expensive operations."

I actually have read the entire article. Have you?

Your explanation has absolutely nothing to do with the performance gains observed. Moreover, in the context of delay-loaded DLLs, your explanation actually makes no sense whatsoever.

Delay loaded DLLs, a linker/loader optimization Microsoft has offered since the days of C++ 6.0 (1998), simply means most process invocations in OP's case won't actually end up loading said DLLs, reducing the amount of time spent in DLL_PROCESS_ATTACH/DLL_PROCESS_DETACH (and specifically during destruction, in the Kernel).

[masklinn]

> I actually have read the entire article.

You provide no evidence of it, and ample evidence to the contrary.

> Delay loaded DLLs, a linker/loader optimization Microsoft has offered since the days of C++ 6.0 (1998), simply means most process invocations in OP's case won't actually end up loading said DLLs, reducing the amount of time spent in DLL_PROCESS_ATTACH/DLL_PROCESS_DETACH.

It also avoids loading gdi.dll, which avoids creating a bunch gdi objects, which avoids taking the "destroy gdi object" codepath on process termination… which is the bit that is both slow and globally serialised.

TFA's final section even demonstrates the difference it makes: 30% increase in start time including 300% increase in lock contention time but 200% increase in shutdown time including 400% increase in lock contention. The process shutdown is almost entirely serialised due to (as TFA and its predecessor explain) a system-wide lock held during GDI cleanup.