[regularfry]

It's probably still too big to be helpful with these model sizes, but if someone helpful runs the same training on `small.en` (and smaller) we might have something.

Yes, this is me praying to the benevolent HN gods that someone will pick this up and run with it. I don't have a GPU anywhere close to capable...

[FL33TW00D]

You'd be surprised how capable old GPUs are! I've had great success with people running Whisper-Turbo in the browser on really old hardware: https://whisper-turbo.com/

[kkielhofner]

We have benchmarks[0] for Willow Inference Server using Whisper + ctranslate2 + some of our own optimizations.

TLD a six year old ~$100 used GTX 1070 is roughly 5x faster than a Threadripper PRO 5955WX at a fraction of the cost and power.

[0] - https://heywillow.io/components/willow-inference-server/#ben...

[RecycledEle]

> TLD a six year old ~$100 used GTX 1070 is roughly 5x faster

Did you mean TIL?

[regularfry]

It's not the inference, it's the training. They say in the paper: "We train with a batch size of 256 for a total of 80,000 optimisation steps, which amounts to eight epochs of training." That's a fair chunk of time. Mind you, `small.en` has smaller decoder layers than `medium.en`...