[somebee]

We've been researching different speech models at Scrimba, and went for Whisper on our own infrastructure. A few days ago I stumbled onto Deepgram, which blows whisper out of the water in terms of speed and accuracy (we need high precision word level timestamps). I thought their claim of being 80x faster than whisper had to be hyperbole, but it turned out to be true for us. Would recommend checking it out for anyone who need performant speech-to-text.

[sason]

80x faster than Whisper is an incredible feat. How is Deepgram's transcription accuracy?

Also, have you heard of Conformer-1 by Assembly-AI[1]? It released a few days ago and supposedly scored higher than Whisper on various benchmarks.

[1]: https://www.assemblyai.com/blog/conformer-1/

[somebee]

In my experience the accuracy is at least a bit better than whisper-small on their enhanced models. But we've just started using it so haven't had time to do many direct comparisons with whisper. Their word-timestamps are _much_ better, which is important if you want to be able to edit the audio based on the transcription.

As for speed I have no idea how they make it so fast, but I'm sure they've written about it somewhere. My guess is at least that they are slicing the audio and parallelising it. Will look into Conformer-1 as well!

[pantalaimon]

The nice thing about whisper is that it runs locally.

[Simon321]

I saw deepgrams claims as well an believed them also, then i tried it, it was TERRIBLE. Don't believe them. It only does well on the benchmark they trained it on. It is faster though but the quality is terrible.

[somebee]

Did you try their enhanced models? We're using it for relatively high-quality audio files and their accuracy is better than the whisper small.en model. More importantly, their word level timestamps is worlds better than whisper.

[abraxas]

Yeah, I'm not sure why people get so hyped up about Whisper. In production use it's middling at best and there are commercial offerings the handily beat it in both accuracy and speed.

Whisper is mostly an academic toy.

[inciampati]

In most real world settings, at least in my personal use, latency to a remote AI comprises most of the usability difficulty with automated speech recognition. The larger whisper models can be run directly on a laptop using multi threading and achieve speech to text transcription that is fully sufficient to almost completely write whole emails, papers, documents with them. In fact, I've written most of this comment using an ASR system on my phone that uses whisper. While the smaller models (like the one user here) can need some correction, the bigger ones are almost perfect. They are both very sufficient and for realtime interactive use I see no future market for paid APIs.

Yesterday I wrote virtually all the prose in the manuscript while walking around with a friend and discussing it. We didn't even look at the phone.

Obviously there's an academic element here because I'm saying I'm using it for writing. But it's more of a human-centric computing thing. I'm replacing a lot of time that my thumbs are spent tapping on keys, my fingers are spent tapping on keyboard, and my eyes are spent staring at the words that are appearing, looking for typographical errors to correct, with time organizing my thoughts in a coherent way that can be spoken and read easily. I'm basically using whisper to create a new way to write that's more fluid, direct, and flows exactly as my speech does. I've tried this for years with all of the various ASR models on all the phones I've had and never been satisfied in the same way.

[DougBTX]

Sounds great! Which app are you using for this?

[inciampati]

"Openai Whisper Keyboard" is good. It doesn't use whisper.cpp but rather a pytorch implementation that runs on Android.

I also use whisper.el on emacs. It's amazing. Much more powerful but computer based of course.

[DougBTX]

Thanks!

[selfhoster11]

Whisper democratises high-quality transcription of the languages I personally care about, whether using a CPU or a GPU. It's FOSS, self-hostable, and very easy to use.

That's a big deal for someone who wants to build something using speech recognition (voice assistant, transcription of audio or video) without resorting to APIs.

[abraxas]

Is this in the realm of aspiration or something you've actually worked on? Because Whisper is incredibly difficult (I'd say impossible) to use in a real time conversational setting. The transcription speed is too slow for interactive use even on a GPU once you step up above tiny or base. And when you step down this low the accuracy is attrocious (especially in noisy settings or with accented voices) and then you have to post process the output with a good NLP to make it usable in whatever actions you're driving.

Look, it's nice that it's out there and free to use. For the sake of my wallet I hope it gets really good. But it isn't competitive with top of the line commercial offerings if you need to ship something today.

[codethief]

At the company I work for we are currently in the process of transitioning from being a German-only to an international company. Recently, we started using Whisper to live-transcribe + translate all our (German) all-hands meetings to English. Yes, it required some fine-tuning (i.e. what size do you choose for the audio chunks you pass to Whisper) but overall it's been working incredibly well – almost flawlessly so. I don't recall which model size we use but it does run on a beefy GPU.

[aftbit]

What if you are operating in an internet-denied application? A remote radio sensing system with a low bandwidth back channel or an airplane cabin, just to name two.

[selfhoster11]

Actively working on it. I've not noticed any performance problems even with the large model (though the plan was always to run the speech recognition on a GPU - your use case may differ). It seems to be doing fairly well even with slightly noisy inputs, and certainly has better bang/$ than other non-API solutions that service my native language.

While true real-time would definitely be nice, I can approximate it well enough with various audio slicing techniques.

[MacsHeadroom]

Faster Whisper is 8x faster than real time on CPU and even faster on GPU. https://github.com/guillaumekln/faster-whisper

Vocode uses Whisper for real-time zero latency voicechat with chatGPT. Give their demo line a call to see how well it works: [phone]

[rolisz]

On a 1080Ti (6 year old GPU) I found Whisper large models to take around as much time to transcribe as the length of the audio.

[inciampati]

That's very similar to CPU-based performance with modern CPUs and parallelization! Frankly, with whisper.cpp it tends to be a little faster than the length of the audio for the "small" model, and much faster for "base" and "tiny".

[pantalaimon]

Doesn't even have to be that modern, my Ivy Bridge CPU already achieves faster than realtime performance - which makes me wonder if there is maybe some upstart cost for the GPU based solution and it would outperform the CPU only with longer clips.

[selfhoster11]

Try quantised models. They perform reasonably well, although you probably want to run some benchmarks if you really want to get it done properly.

[dontreact]

If there is a better mode I can easily integrate using python I’m all ears. For what I’m building quality is most important

[abraxas]

Oh, Azure's speech recognition API beats it handily on English language. Both in accuracy and speed.

Another is Deepgram. Even this obscure vendor seems to be able to handle the samples I tried better than Whisper: https://picovoice.ai/platform/cat/

But yeah, go with Azure as your starting point. It is good and the price is likely acceptable unless you're transcribing all of youtube.

[dontreact]

Umm I want to pay zero and run locally

[generalizations]

If you use the large_v2 version of whisper, and give it a prompt to indicate what it's transcribing, it can do extremely well. But do use the prompt feature.

[dontreact]

Yeah exactly this is why there’s hype. It’s the best model that you can use for free easily

[leetharris]

I work at Rev.AI, check us out

Lowest WER in the industry, cheaper than Whisper API, and we have an on-prem solution