The coherent memory interconnects between nodes is typically what makes supercomputers different than just a bunch of consumer hardware. It allows different types of programming or at least makes them easier.
It's a very fast, very low latency network fabric. But it's not coherent in the sense of cache coherent multiprocessors, and it doesn't offer shared memory style programming where you'd just load/store to addresses that happen to be mapped to another compute node somewhere in the system.