From what I remember of Infiniband, the network cards are able to do direct PCIe transfers to/from compatible devices attached to the host (eg nvidia quadro gpus).
precisely this. The reason why its worth the expense is that data entering the network is guaranteed to come out the otherside. This kinda eliminates the need for TCP (apart from addressing)