Downside: numpy, h5py and others use an embedded libhdf5 which is quite outdated. Meanwhile, homebrew has the current version, but they ignore it. In contrast, the Debian/Ubuntu/FreeBSD ports have python packages built against the packaged libhdf5. The pip packaging is inferior to the properly packaged stuff, which is what you get when language-specific packaging exists in its own isolated bubble.
I feel like the correct move here is to push the library maintainers to update to the new version. Distribution package mods should really be seen as temporary.
I have already opened an issue for this. However, it doesn't resolve the underlying and rather harder to fix problem of different language-specific package managers embedding copies of C and C++ libraries which can end up in a horrible incompatible mess. It's not specific to pip/pypy, it also applies to Java jars with embedded libraries and other systems which also embed.
Sometimes, this embedded copy can be more up to date than the system copy, but the convenience of embedding is often outweighed by the binary compatibility problems and being neglected and out of date.