The real problem tends to be the (CPU to other thing and back again) latency, not the how fast can the other thing do the computation.