This is true but keep in mind that once latency starts to go over ~10ms it starts to become noticeable for a lot of people, and even if it isn't directly noticeable it's likely that VR latency still causes some physiological issues for most people.
I believe it's more the processing that would be needed to send the data over the larger distance in fewer wires which would add latency.
You can't just take the absurd number of traces off a PCB and shove them into a wire without hitting issues like voltage drop, crosstalk, interference, or having a cable as thick as your finger.
You'd need optical transceivers and at that point the added complexity and similar board space required might negate any weight savings (and drive up the price further).