[aappleby]

Why even bother with the GPU at that point? CPU would be just as fast if you're bottlenecked on SSD bandwidth.

[anuarsh]

CPU is much slower than GPU. You can actually use both by offloading some layers to CPU as o.offload_layers_to_cpu(layers_num=12). It is faster to load from RAM than from SSD.