[regularfry]

We really, really need something to take Whisper's crown for streaming. Faster-whisper is great, but Whisper itself was never built for real-time use.

For this demo to be real-time, it relies on having a beefy enough GPU that it can push 30 seconds of audio through one of the more capable (therefore bigger) models in a couple of hundred milliseconds. It's basically throwing hardware at the problem to paper over the fact that Whisper is just the wrong architecture.

Don't get me wrong, it's great where it's great, but that's just not streaming.