What speech recognition system did you use on Linux? I'm interested in using speech recognition more after getting used to it on iOS devices (where it works amazingly well).
I had the most success using Dragonfly with the Kaldi engine, which handles the heavy lifting of speech recognition and provides a nice API on top of xdotool for desktop interaction. You can also check out Talon, although I think its Linux support is still somewhat experimental.
Thank you. These keywords (Kaldi, Dragonfly) put me on the right track and I found nerd-dictation [1] which is easy to use and hackable. Perhaps not useful for your programming purposes, but I need something for writing prose and it's a decent start for that.