I've never been terribly bothered by compile times in general. 90% of my work is with compiled languages, and (in most languages) incremental compilation does a great job of keeping compile times low. Even for large projects on slow machines, I rarely see compile times exceed even a minute. More usually, compiling just takes a few seconds.
Sure, you do occasionally need to do a full recompile, and if the project is very large then this can take a nontrivial amount of time, but that doesn't really happen often enough for me to be bothered about.
If the project is of really significant size, or if the compiler/language/code structure is such that incremental compilation can't give you significant gains (or isn't even possible), you may need more grunt of course.
Sure, you do occasionally need to do a full recompile, and if the project is very large then this can take a nontrivial amount of time, but that doesn't really happen often enough for me to be bothered about.
If the project is of really significant size, or if the compiler/language/code structure is such that incremental compilation can't give you significant gains (or isn't even possible), you may need more grunt of course.