[modeless]

The hardware that matters here is not the GPU. It's the touchscreen controller, the display controller, the associated firmware, and to a lesser extent the drivers for those. All either Apple proprietary or heavily customized.

[mdwrigh2]

The GPU can matter as well. On the Nexus 10, for example, the fill rate is pretty low compared to the screen size, so applications have to be very very careful not to exceed the limit or they'll end up adding frames of latency (which looks like touch or interaction latency).

But in general you're right, those are the components that contribute the large majority touch latency to a device.