This is where IDEs and sbt's ~compile come in handy. I don't hit build very often, since the IDE tells me about my mistakes interactively, and, once it's happy, a build makes for a great micro-break :-)
I agree that for code changes, incremental compilation is usually reasonably painless. Compile times really kill me when I have to do a lot of clean builds, like when modifying my build scripts.
One of the things I was hoping for in improved scala tooling was better IDE agnostic solutions for developers.
For instance, I much prefer to use vim than an IDE, but due to the near requirement to have syntax checking in the editor to avoid long compile cycles I always end up giving up and using an IDE. That a common response to slow tooling complaints is that an IDE hides these issues is an indictment of the scala tool chain.