[solarkraft]

Oh, I like this! I’ve been looking into locally hosting a transcription API server and came away feeling pretty close to the problem statement. The things most frequently lacking were streaming support (which I’m so glad this has!) and the support for special words to boost during recognition (which I guess there’s some hope they might add???).

[embedding-shape]

> I’ve been looking into locally hosting a transcription API server

I've been hosting my own since whisper.cpp appeared on the scene, thrown up on a server with a 3090ti. Even if there is better/faster stuff out today, it just keeps on working without any issues, the weights are tiny and it's faster than I could need. This is basically what you need to get this working today:

    MODEL="/home/user/projects/ggml-org/whisper.cpp/models/ggml-large-v3-turbo.bin"
    WHISPER_SERVER_BIN="/home/user/projects/ggml-org/whisper.cpp/build/bin/whisper-server"
    "$WHISPER_SERVER_BIN" --model "$MODEL" --language en --host 127.0.0.1 --port 7812

Very simple stuff, throw it on some local homelab server and now you have a local transcription API :) Might need to play around with some of the inference parameters, but once you've locked them in, seems to work really well.

[solarkraft]

Does it support streaming? I find that this is the #1 thing missing from almost all implementations.

[sipjca]

word boosting will probably come on a much longer time horizon, but streaming is here!

I'm really hoping someone either contributes a good server example to the codebase (and is willing to help with issues) or use transcribe.cpp or the bindings to create a robust server in another language :) would be happy to link it from the main project directly as well