[user5994461]

My guess is the impact of cross site caching is negligible. We're losing nothing here.

1) Cache hit must be extremely low because of different versions/variants/CDN for each library. (Have you seen how many jquery there are?).

2) It's irrelevant outside of the few most popular libraries on the planet, maybe jquery/bootstrap/googlefonts.

3) Content is cached once on page load and the saving are happening over the next tens or hundreds pages you visit. That's where the gain of caching is (10x - 100x). Saving 1 load when changing site is negligible in the grand scheme of things.