Hacker News new | ask | show | jobs
by gmueckl 572 days ago
If you run many small tasks on the GPU, you can increase throughput by overlapping transfers and computation. There may also be other ways to batch problems together, but that depends on the algorithms.

The one truly unfixable issue is round-trip latency.