[mhh__]

Firing up a GPU just to do a little bit of inference sounds quite expensive.

CPUs (let alone CPUs talking to a GPU) spend huge numbers of cycles shunting data around already.