Yields of functioning chips quickly drop off when you increase the size, because each single transistor has an independent risk of error. Individual error rates are extremely small, but so the number of possibly failing components is large.
You don't need every transistor to work though. If you can detect when a module has a broken transistor, you can simply disable that module and sell the rest of the chip. Divide the cache into multiple small modules, and it is not a big deal if you have to deactivate one because of a broken transistor. You would probably be deactivating it anyway for market segmentation.