https://github.com/ggerganov/whisper.cpp
Which uses x8 less memory than the Python implementation for the tiny model. It would be a good idea to keep an eye on it since there are Python bindings planned on the roadmap:
https://github.com/ggerganov/whisper.cpp#bindings
https://github.com/Const-me/Whisper
https://github.com/kardianos/audioclerk
Built in Go/cgo.
https://github.com/Const-me/Whisper