It's actually really easy in Claude Code. Get a TUI to the point where it renders something, and get Claude to the point where it knows what you want to render (draw it in ASCII like this post proposes, for instance).
Then just prompt Claude to "use tmux to interact with and test the TUI rendering", prompt it through anything it gets hung up on (for instance, you might remind Claude that it can create a tmux pane with fixed size, or that tmux has a capture-pane feature to dump the contents of a view). Claude already knows a bunch about tmux.
Once it gets anything useful done, ask it to "write a subagent definition for a TUI tester that uses tmux to exercise a TUI and test its rendering, layout, and interaction behavior".
Save that subagent definition, and now Claude can do closed-loop visual and interactive testing of its own TUI development.
Can you explain tmux's contribution here? I'm confused why this process wouldn't work just the same if CC directly executed the program rather than involving tmux. Are you just using tmux to trick the program under test into running its TUI instead of operating in a dumb-stdout mode?
I had a really hard time getting anything like that to work (you can't just read stdout and write stdin, because you're driving a terminal in raw mode), but it took like 3 sentences worth of Claude prompt to get Claude to use tmux to do this reliably.
I tell Claude code to use an existing tmux session to interact with eg a rails console, and it uses tmux send-keys and capture-pane for IO. It gets tripped up if a pager is invoked, but otherwise it works pretty well. Didn’t occur to me to tell it to take screenshots.
Also many CLIs act differently when invoked connected to a terminal (TUI/interactive) vs not. So you’d run into issues there where Claude could only test the non-interactive things.
"In essence" but terminals do stuff to render stdout that you do not want a LLM to have to replicate, I think. If your TUI does stuff in fullscreen or otherwise with a bunch of control codes, that is simple work for a terminal but potentially intractable for a LLM.
Then just prompt Claude to "use tmux to interact with and test the TUI rendering", prompt it through anything it gets hung up on (for instance, you might remind Claude that it can create a tmux pane with fixed size, or that tmux has a capture-pane feature to dump the contents of a view). Claude already knows a bunch about tmux.
Once it gets anything useful done, ask it to "write a subagent definition for a TUI tester that uses tmux to exercise a TUI and test its rendering, layout, and interaction behavior".
Save that subagent definition, and now Claude can do closed-loop visual and interactive testing of its own TUI development.