[fluffything]

Is there a blog post about Visual C++ being multi-threaded end-to-end ? (e.g. doing parsing, type-checking, etc. of a single TU in parallel using all cores ?)

That would be super interesting to read because it is mainly a C++ compiler and some C++ features like two-phase lookup and macros make it quite hard to do things in parallel. You have to do things in a certain order but I suppose that if its query-based as well it will work.

The only option I know is /CGTHREADS but that only uses multiple-threads for optimizations and code generation which is something that the Rust compiler has been able to do for a very long time (e.g. there these are called codegen-units and LLVM supports these so it is quite trivial for a frontend to do so as well).

[zone411]

As the documentation says: "The /MP (Build with Multiple Processes) compiler flag specifies the number of cl.exe processes that simultaneously compile the source files. The /cgthreads option specifies the number of threads used by each cl.exe process." I've never used /CGTHREADS but /MP is working well (except for the precompliled header file).

[fluffything]

The documentation also says that /cgthreads is the number of threads used by the code generation and optimization passes of the compiler, not by the whole compiler.

A compiler needs to do a lot of stuff beyond code generation and optimizations (e.g. in debug builds optimizations are even often disabled). For C++ you have overload resolution, template argument deduction, template instantiation, constexpr evaluation... and well parsing, tokenization, type checking, static analysis (e.g. for warnings), etc.

Parallel optimizations and codegen is trivial when compared with an end-to-end multi-threaded compiler. All LLVM frontends for all programming languages do parallel optimizations and parallel codegen, it's only an LLVM option away. You enable it, and it's done.

> The /MP (Build with Multiple Processes)

This is one process per translation unit, each single translation unit is then compiled with a single thread until optimizations and codegen.

Often you need to finish compiling/linking some translation units before continuing.

The Rust compiler has pipelined compilation: compilation of a translation unit starts as soon as what this requires of its dependencies is already available, before its dependencies have finished compiling. It also compiles each translation unit itself using multiple threads end-to-end, so that if one of your translation units is bigger than the others, you can speed the compilation of that one by throwing more threads at it.

[virgilp]

That's for different source files though, right? Try having a 10Mb auto-generated C++ file, see how fast it compiles, regardless how powerful your computer is.

[fluffything]

No need to autogenerate anything: expand all headerfiles into a .cpp file, and you easily end up with > 10Mb files.

Visual C++ pre-compiled headers is quite amazing and works really great, avoiding this almost completely.

[pjmlp]

Visual C++ MSDN blog has a couple of blog entries how they refactored the compiler to use AST, added incremental compilation and incremental linking.

But as far as I remember it isn't fully multi-threaded across all phases.