You would lose the ability to do transparent caching which I agree is rather annoying, but I think most environments where that sort of caching occurs (mostly corporate and school networks) also have the means to explicitly configure client machines to use an internal caching mirror.