[rasz]

OnLive "solved" encoder latency 15 years ago. You dont wait 16ms for the next frame. Instead you progressively start encoding after receiving first tens of lines. This way your encoded video stream lags just couple of milliseconds behind, same for decoding. You could crudely emulate this by dividing screen into 4 rows and sending 4 concurrent video stream, instant 1/4 latency drop.

[sephamorr]

Sure, many of the operations in the list can be pipelined as you mention. Something like G-Sync would also allow you to sync the destination display to the arrival of the (start of) frame.