[earthtolazlo]

I had the most success using Dragonfly with the Kaldi engine, which handles the heavy lifting of speech recognition and provides a nice API on top of xdotool for desktop interaction. You can also check out Talon, although I think its Linux support is still somewhat experimental.

[tmalsburg2]

Thank you. These keywords (Kaldi, Dragonfly) put me on the right track and I found nerd-dictation [1] which is easy to use and hackable. Perhaps not useful for your programming purposes, but I need something for writing prose and it's a decent start for that.

[1] https://github.com/ideasman42/nerd-dictation