|
|
|
|
|
by docstryder
38 days ago
|
|
Some one asked this but got flagged so still answering it here: ejoso 2 hours ago: The latency breakdown is honest and the right frame. Most of the delay budget gets eaten before and after the model, and that is harder to fix than it looks. Cold starts on local Whisper variants and WebSocket warmup on cloud paths are both worse than benchmarks suggest.
The differentiation question I keep landing on: whisper.cpp behind a hotkey with a paste shim is a solved afternoon project for a certain kind of person. The real gap is consistency across arbitrary apps, and that is genuinely harder than it sounds. For the fully local path (flying, privacy-sensitive) what does Shoute add beyond well-packaged WhisperKit with better insertion handling? That answer is either the core pitch or an honest scoping of who this is for. ------------------------------------------------------ This makes a good point - latency and consistency are indeed the hardest to get right.
For the fully local path - the major value add for Shoute is still the speed and consistency, getting accessibility settings right and the flow from the model to text is useful for many folks. That and consistently keeping it updated with model updates as models improve and with OS updates. |
|