I think so. It would be a bit more involved that "replace the package for gcc-11". The packages are addressed by the inputs which built them.
There has been some effort towards supporting content-addressable storage. https://www.tweag.io/blog/2020-09-10-nix-cas/ https://edolstra.github.io/pubs/secsharing-ase2005-final.pdf