Hacker News new | ask | show | jobs
by ramesh31 1006 days ago
Offloading 40% of layers to CPU, about 50t/s with 16 threads.
2 comments
pocketarc 1006 days ago
That is more than an order of magnitude better than my experience; I get around 2 t/s with similar hardware. I had also seen others reporting similar figures to mine so I assumed it was normal. Is there a secret to what you're doing?
link
ramesh31 1006 days ago
>Is there a secret to what you're doing?

Core speed and memory bandwidth matter a lot. This is on a Ryzen 7950 with DDR5.

link
jpdus 1006 days ago
Care to share your detailed stack and command to reach 50t/s? I also have a 7950 with DDR 5 and I don't even get 50 t/s on my two RTX 4090s....
link