[saulrh]

If I were building a terminal from scratch... Hm.

Attempts to improve the terminal are not backwards-compatible or do not otherwise provide a clean upgrade path. I would love to use one of those fancy graphical terms that comes up on HN every now and then, but because they only work with a special toolchain that speaks its language and its language only there's no way they'll ever gain critical mass and take off. Any attempt to improve the terminal has to be something that I could switch to on a whim and then gradually grow into.

I really like the "curious fact" from higher in this thread and I think that it's important. Whatever the solution is, I think that it'll necessarily involve separating "business" logic from presentation logic, and giving presentation logic to the terminal while things under the shell handle the actual work. Right now, one of the biggest problems with writing a CLI tool is that TUI work is a huge pain in the ass. It's harder to make a table in the terminal than it is to make a table on the web! That's insane! And that suggests a straightforward solution: Move display logic into the terminal so that it can handle layout and rendering. And you do that the same way that that's been done in literally every situation that's ever been done: You ship a small program that constructs a view that can take advantage of a more favorable execution environment.

So what does it mean to "ship a program that constructs a view"? Well, even if your program is written in a deeply standardized, weak language (e.g. ANSI escape codes) it's still a program. Formal languages theory, hooray. So we're already operating in this formalism, which makes our job much easier and the solution relatively obvious - Instead of inventing an entirely new paradigm from the ground up we just need to jam a more powerful language into an unused corner of the existing one.

This is where I go down the rabbit hole, because I don't know enough about the specifics of ANSI escape sequences. My goal, essentially speaking, is to find a set of sequences of control characters (a language) that will be ignored by everything except my New Terminal, map that set to some convenient programming language (ideally by a trivial "prefix-identity-suffix" construction), build a platform for building user interfaces that uses that programming language, and then build... something that augments existing programs with New Terminal front-ends? A lump of stuff into my shell RCs that detects New Terminal and augments the prompt strings with New Terminal apps that wrap the outputs of commands that look right?

Then the question is, what does my platform look like, and what convenient programming language am I using? I'd probably go with something like js or guile. Maybe just js so I can take advantage of extensive existing UI tooling. I mean, if you look at this sufficiently sideways, I've essentially duplicated the evolution of HTML+JS webapps - jam a new language (js script tags) into a corner of the existing language (html) that will be ignored by systems that don't understand the new thing and then implement a platform (DOM manipulation and such) that allows the creation of fancy user interfaces (webapps) using those tools. And really, I think that that's the clincher - that evolution happened when client machines became powerful enough that it was useful to move presentation logic out of the server and into the client, and that's exactly what we need to do with terminals.

(thought: interaction. callbacks return a sequence of characters that're typed into the terminal? so you can click on a table header and the client sorts and re-renders the table, or you can click on a button and to run a convenient follow-up invocation? ahhh, yeah - if the backend program (ls et al) is transient its interactions will render commands for follow-up operations, if it's persistent its interactions will render keystrokes for interacting with it - you'd click on the headers in top and it'd send `p`, `t`, etc. that's really clean.)

[terminal-survey]

> I would love to use one of those fancy graphical terms that comes up on HN every now and then

One of these (Black Screen) is among the many inspirations with which we started.

> Any attempt to improve the terminal has to be something that I could switch to on a whim and then gradually grow into.

Backwards-compatibility is a hard requirement for us because of that. And oh boy, it causes a lot of pain.

> I've essentially duplicated the evolution of HTML+JS webapps

This is a huge fear of mine. We need to find a way of opening the GUI can-of-worms without ending up with terminals that are as bloated as contemporary web browsers.

[saulrh]

> Backwards-compatibility is a hard requirement for us because of that. And oh boy, it causes a lot of pain.

Yeah, it's a bitch. Sorry. :( I've seen one or two "rewrite everything and make it better enough that people switch" big-bang efforts succeed - I run fish, 'nuff said - but this is a rather more invasive change, and it doesn't have good fallbacks ("run bash for ten seconds").

I'm not sure there is a good solution here, not when I look at it from the formal languages perspective. The existing language (what's relevant to current functionality) contains nearly all sequences of elements of the available alphabet (bytes), so anything we do will collide with something. Nobody was able to leave a specific exception to permit extension the way HTML did when it decided to silently ignore unrecognized tags. Though maybe the ANSI "application program command" escape codes might be usable? Either way, this really seems like a question of what you're going to have to choose to break, and that's never fun.

> This is a huge fear of mine. We need to find a way of opening the GUI can-of-worms without ending up with terminals that are as bloated as contemporary web browsers.

Pretty much. You have a titanic advantage, since you don't have to deal with styles at all and can probably start over with a non-hierarchical layout based on integer constraint satisfaction, but you're still stuck trying to bootstrap a small OS/environment. Maybe "only" the size of something like a well-designed video game modding API (factorio, kerbal space program, MMOs, etc).

Heh, emacs might be another thing you could look at, they're having some of these issues themselves.

------------------------------

What are the "killer apps" for these features? `cat foo.jpg` is a parlor trick; what could you do that'd actually make me want to switch? We'd probably be looking for small but effective quality-of-life things that anyone can use - for example, fish-shell got me with "it just works" completion and syntax highlighting.

Hmm. Every tabular output (ps, ls, df, docker, random stuff massaged with awk and cut) magically having sortable columns in the original data, no rerunning the command with a different flag? Parse `--help` to make a checkbox/textbox for each flag and a "do it" button that assembles the resulting invocation? A standardized set of "quick invocations" that a command can give to the terminal to be quickly invoked with something like `M-[[:digit:]]`, so that, say, the suggestions from `command-not-found` would be a single keystroke away, or ls could provide "cd to directory 1, 2, 3"?