[tbirdz]

It's not the bandwidth that's the problem, it's the latency. Many problems need more control flow and branching, and that is better done on the CPU. If you need to make decisions and take the previous iterations output as an input, then the overhead of transferring the data back and forth from cpu to gpu outweighs the benefit of the gpu speed.

GPUs are good if you have an independent data parallel algorithm you are using to transform large blocks of floating point data. For other uses, CPU is better.