Up to a point, eventually when the code base is large enough you spend more time serialising/deserialising and passing things between workers that it can be faster to disable the cache.
From experience with one of the later betas (around 22 or 23?) we saw no improvement or slight regression with the disk cache vs the memory cache for incremental builds. Initial builds (warm starts) were faster, but there was still significant overhead reading the cache - this is an area where any optimisation would yield big results. I haven't had the chance to try the released version yet, but my understanding is there weren't any significant changes in this area.