[fulafel]

I think you are working with an oversimplified model of graphics programming if you think graphics code runs just on the GPU.

But even in this ideal scenario, there would already big sacrifices involved in just offloading this stuff to GPUs: Sacrifices in SW complexity and development time moving from programmer-friendly CPU code to hard-to-program GPU interfaces.

The reality is in fact a CPU+GPU juggling setup where interacting CPU and GPU computations must be carefully coordinated and performance-tuned.

If the CPU bottleneck didn't exist and we could do without today's GPUs, we could make better graphics applications (games) given the same investment and technical engineering competence, and game developer life would be much more pleasant.

(There's an alternate history of more programmer friendly GPUs of course too, see eg the Larrabee story)