Hacker News new | ask | show | jobs
by tsbinz 2061 days ago
Note that these are light models that are designed to be run quickly on a cpu with batch size 1. It's not that uncommon to see multithreaded cpu code beat the gpu in that setting also for other backends.