[GhostVII]

I feel like we need a complete rewrite of terminal emulators/bash/whatever. It should be super easy to make a CLI with nice colors, good loading icons, etc. without having to deal with all kinds of color codes and cursor movement. When I press "enter" while a script is showing some progress bar, or resize the terminal window, it should handle it nicely like every other application.

I use the command line whenever possible over a GUI application, but sometimes I wish it wasn't still stuck in the 80s. Keep the speed an simplicity (I don't want images or anything), but make it easy to make robust applications with a clean text based UI. I shouldn't still have ghosting in my Vim setup, it's 2020.

[mulmen]

I kinda like using tooling with 40 years of refinement. Is it perfect? No. Do I need to resize a window with a progress bar? No. Would it be nice? Yeah, I guess. Am I underestimating the benefits? Almost certainly.

But what are the other sharp edges that come with the shiny new thing? Would it be an electron app? Yes, probably. Would it work? Maybe, in a couple years. There would be some major bugs first, and a lot of dead batteries.

I don’t think new devs are specifically worse than old ones. I do think there is immense value in using something that has been actively improved for your (well, at least my) entire lifetime.

There are lessons buried in there we don’t even begin to understand. Starting over means forgetting those lessons too.

The grass is always greener on the other side of the fence is what I am saying.

[michaelmure]

It looks to me that the reality is closer to "40 years of hack over something that was not meant to be a standard (a popular hardware terminal from the 80s)".

I'll give you just one example of the madness that is sometimes required: the escape code \e[1m is doing "bold on". It's counterpart to reset \e[21m, is supposed to do "bold off", but in practice some terminals do "double underline" instead because this standard is not a real standard. As there is also no real way to detect capability, the only reliable way to do "bold off" that I could find is to intercept the stream of data sent to the terminal to compute the attribute state. When you want to do a "bold off" you do a \e[0m "reset all" instead and sent the control character to set the state again without bold [1].

A new standard would also allow to reliably use features like displaying images or links.

[1] https://github.com/MichaelMure/go-term-markdown/blob/master/...

[spijdar]

So, what you're saying is, since there are too many competing and incompatible standards, what we need is another incompatible standard? :)

There are a lot of "better" terminal emulator standards out there. The problem is you need software that uses them. Perhaps this is the real "refinement" spoken of before -- these quirks are generally understood and software layers are built on top of them to abstract away the ugliness while keeping broad compatibility.

There are lots of ideas on how to make a better CLI -- from Plan9's rio/window to various XML/DOM based terminal emulator, to a smorgasbord of xterm-but-added-features emulators that add support for anything from inline images to transmitting entire files over the TTY. Ultimately, these are only as useful as the software that uses them, which often ends up being just a few tools the terminal emulator itself ships with.

Even Windows' alternative to TTYs has basically been supplanted by an xterm/VT-ish compatible TTY style interface for software compatibility reasons.

[oddlama]

Relevant xkcd: https://xkcd.com/927/

[spc476]

I hate to say it, but the standard for C1 escape codes, ECMA-48, does list "\e[21m" as "double underline". I can see where one might think it does a "bold off" but alas, that isn't in the standard.

The standard also has a lot of redundancy, and some omissions (getting the size, but I think it was standardized long before TTYs were resizable).

A standard is a good idea, but I think ECMA-48 was a draft that was put into production too quickly.

[michaelmure]

My point exactly. Some terminals do the expected thing of doing "bold off" because that make the mapping of codes regular, some follow ECMA-48 and do "double underline" even if that means that "bold off" is not possible.

At the end, there is no way to reliably do either of them.

[IncRnd]

Not using bold properly means the terminal was written incorrectly, or the wrong termcap is being used to emulate a different terminal.

This isn't a bug with escape codes or with terminals that work. It is either that someone doesn't know which teminal they are using, or there is a bug in that specific terminal.

[digisign]

Not entirely. Well there is bold, normal, and dim. Early on someone realized there is no need for unbold and undim codes. So undim, 22 I believe, works for both, setting normal intensity. 21 was then free for something else.

[thih9]

I’d say all this is a feature.

This means that it’s difficult to write a flashy app with bold text, links and images.

I like programs with minimal text ui, so I don’t mind.

[michaelmure]

A new standard would also make it easier to fallback to no colors/decoration if the user wish so. At the moment, there is no way to do that without a flags or similar for each app. Even detecting if the terminal support colors is a pain.

[thih9]

> fallback to no colors/decoration if the user wish so

Fallbacks are frequently not implemented, not tested or not supported as well as the main feature.

> Even detecting if the terminal support colors is a pain.

As the end user, I don’t mind. This means that I’ll see colors only when it’s important.

[kjeetgill]

I'm with you 100%, virtually nobody will take the time to code against a fallback profile like that. That's why most sites tend not to work without JavaScript regardless of whether their functionality strictly requires it.

Support for images concern many of us for that reason: sure it seems nice for a few cases used judiciously. But it feels counter-productive to the strengths of the CLI.

I would feel that way about curses/cli-based-gui too in theory, but in practice it's just top, vim, and less for the most part. Maybe I'm just too scared.

[vkou]

> I kinda like using tooling with 40 years of refinement

I would not call the modern terminal the result of 40 years of refinement - mostly for the reasons outlined by the parent poster, as well as those of the sibling posters.

If you look at the modern terminal through fresh eyes, 'buggy and inconsistent and clunky' would be how the average homo sapiens would describe it.

Just because there are reasons for why all of these warts are in place, doesn't mean that the program is not defective, as designed.

[cuddlecake]

There are a lot of "Maybe"s in your second paragraph.

I'd say, yes, there are a lot of lessons buried in old code, but often, there is an equal amount of baggage we are too fearful to abandon because of backward compatibility and "bug became feature"-situations.

And even then, it's not you who is abandoning that, which makes the attitude even more baffling to me.

[craftkiller]

There still some pretty glaring issues, even after 40 years of refinement. Some I think could be iteratively fixed but not all of them (for example I believe #1 requires a significant breaking change to terminals).

1. You can't have an interactive program in a pipeline. For example, you can't do `gpg --decrypt foo.txt.gpg | nano | gpg --encrypt` because keyboard input on the console is handled through stdin. You can work around this by using a graphical editor in the pipeline, but this should be possible entirely over the command line.

2. You can't have hotkeys that are just modifier keys because the modifier keys only change the byte sent when pressing another key. So could couldn't have a hotkey "Shift" but you can have a hotkey "Shift-n".

3. How do you tell the difference between an escape sequence and the escape key? Escape sequences start with exactly the same byte used by the escape key

4. While we have escape sequences to move the cursor and detect the cursor location, in order to find dimensions of the terminal we have to resort to an ioctl.

5. Termcap. Because we can't assume some standard set of terminal features, and since my FreeBSD servers don't have a termcap file for alacritty, when I ssh into my FreeBSD servers my backspace key doesn't work (along with a lot of other things).

6. This ones a bit optional, but it would be nice to have a standard protocol for telling the terminal to render an image.

7. Abruptly killing a program that has put the terminal into raw mode leaves your terminal in raw mode.

What would I do to fix this:

1. Move keyboard input to a different file handle than pipes. This would require software to be patched, or at least a compatibility layer written to merge the pipes for legacy software.

2. Remap all the keyboard bytes. Every key press and every key up sends data to the keyboard input stream. Let the program maintain a keyup/keydown state map for handling modifier keys. This would require software to be patched or at least a compatibility layer written to remap the bytes.

3. As part of the new encoding for #2, we should make a better encoding for input. The current system seems a lot more "grown" than designed. I'd love to see something similar to UTF-8 where we get to encode a variable-length integer with the benefits of indicating byte length in the first byte and being able to detect if we're in a continuation byte. We could even use UTF-8 verbatim for most of the keyboard input but we'd need a special section for escape sequences like moving the cursor. I don't know enough about unicode to know if theres a section we could use for that, but worst case scenario we could use the 5 and 6-byte length utf-8 sequences that never made it to the final utf-8 spec despite being possible in the encoding.

4. As part of #3, add escape sequences to detect terminal dimensions.

5. Establish a new minimum set of features that are assumed to work, incorporating all these refinements over the past 40 years.

6. This could be part of the new encoding for #3

7. Automatically revert terminal to its original non-raw-mode state at the end of a program.

Finally, I'd recommend this blog post to appreciate how much of a mess terminal programming is: https://viewsourcecode.org/snaptoken/kilo/02.enteringRawMode...

[skissane]

> 2. You can't have hotkeys that are just modifier keys because the modifier keys only change the byte sent when pressing another key. So could couldn't have a hotkey "Shift" but you can have a hotkey "Shift-n".

Later model physical DEC terminals had a mode called PCTERM (with escape sequences to enter/exit it) in which the terminal would send PC keyboard scan codes for key presses. However, very few terminal emulators implement that.

The Telnet server and client in Windows NT implements a somewhat similar feature, the VTNT terminal type, in which keyboard scan codes and mouse movements are transmitted in a binary format which is based on the Windows console API. (Nowadays Telnet is mostly deprecated in favour of SSH; there is no reason why SSH clients and servers could not implement VTNT, but I haven't seen any do it.)

The technology already exists to solve this problem, and has for years. It is just that nobody appears to care about it enough for that technology to become widely adopted.

[a1369209993]

> For example, you can't do `gpg --decrypt foo.txt.gpg | nano | gpg --encrypt` because keyboard input on the console is handled through stdin.

I actually had a alias at one point that did something along the lines of (IIRC):

  | editor | -> | editor -i /dev/fd/3 -o /dev/fd/4 3<&0 4>&1 0<&2 1>&2 |

Since stderr (aka &2 aka /dev/pts/whatever) is opened read-write, you can dup it onto stdin to recover the terminal. Ideally, every editor should just support reading from stdin, interacting via stderr[0], and writing to stdout as a command line option though.

Edit: 0: or /dev/tty as jclulow suggested.

[jclulow]

With respect to interactive programs in a pipeline, this can already work today if the software is written to allow it. While it is common for interactive software to use the standard descriptors for I/O they are not required to. You can instead open the magic device, /dev/tty, which gets you new handles to the controlling terminal for the process.

[eitau_1]

Here's a proposal to fix #2 and #3: http://www.leonerd.org.uk/hacks/fixterms/

[Mikhail_Edoshin]

1. You can have an interactive program in the pipeline, for example 'less' is usually called this way:

  cat some.file | less

Less reads 'stdin' and then reaches out to the actual console to respond to navigation keys. The console is reachable either via '/dev/tty' (POSIX) or, on Windows, via CreateFile()[1].

But yes, most programming tutorials (and probably many programs) make no distinction between the console and standard streams, hence the confusion. They absolutely should.

[1] https://docs.microsoft.com/en-us/windows/win32/api/fileapi/n...

[bacon_waffle]

> since my FreeBSD servers don't have a termcap file for alacritty, when I ssh into my FreeBSD servers my backspace key doesn't work (along with a lot of other things).

A hack I've got in .zshrc for this problem:

  function ssh {
   if [[ "${TERM}" = alacritty ]]; then
    env TERM=xterm-256color /usr/bin/ssh "$@"
   else
    /usr/bin/ssh "$@"
   fi
  }

[ac29]

Installing the terminfo also works: https://github.com/alacritty/alacritty/blob/master/INSTALL.m...

[fiddlerwoaroof]

Yeah, I create a .terminfo directory in my home folder on any box I ssh to and make sure it has the terminfo for the terminal I use.

[spc476]

For issue #3, it would have been nice had the TTY layer sent DLE ESC when someone hits the ESC key. As it is now, once you read the ESC key, you have to wait some number of milliseconds to see if more data is coming in, and if not, then just return a single key, else it's an escape sequence. Sigh.

[dmitryminkovsky]

I definitely enjoy the continuity, call me small-minded but I don’t want to see this stuff revolutionized either. Maybe because I came up in the 90s and am nostalgic for the VT220 etc, I don’t know. And it’s weird because I have little or no love for other old school stuff.

[nomel]

I would be happy if I never had to write a flaky text parser, for some command line tool, output again.

[fiddlerwoaroof]

Then don’t, take json or some standard format with an off-the-shelf parser as stdin and write it as standard out. Then just use jq to munge data into json on one end and back out on the other.

[nomel]

> with an off-the-shelf parser as stdin

These only exist for well known tools. And, these usually explode if you're not careful about the characters in your filename.

[fiddlerwoaroof]

That’s not what I’m saying: your program expects to receive standard input in json format. And uses your favorite JSON parsing library to handle stdin. Generate stdout as JSON with the same library. Then, use jq to generate the standard input and output.

[eitau_1]

I treated myself a deep dive into how CLIs work some time ago and features like:

* marking output from stderr different color (and don't interfere with commands's own formatting much);

* passing more involved keyboard shortcuts to the shell or a tui program, etc.;

* making last line of last command usable in next command or putting it to clipboard without mouse;

* general awareness of shell/terminal emulator of what's displayed (prompt, edited line, commands printing in parallel, stdout vs stderr, messages of shell (jobs))

are very tricky to achieve without reorganizing the whole stack of terminal emulator, shell, line discipline/readline and even kernel.

[0] http://www.linusakesson.net/programming/tty/index.php

[1] https://utcc.utoronto.ca/~cks/space/blog/unix/TTYLineDiscipl...

[2] https://github.com/sickill/stderred

[3] http://www.leonerd.org.uk/hacks/fixterms

[4] https://domterm.org/Wire-byte-protocol.html

[rozab]

I've spent a lot of time messing with terminal UIs, and I feel that the experience could be improved hugely just by adding a couple new escape sequences, for right aligned text and for filling to the rest of the width of the terminal with a character. Thanks to terminfo this shouldn't affect backwards compatibility.

I certainly don't think a full rewrite is necessary (we managed to tack on mouse support and even image rendering[0] no problem). Curses is a pain to work with directly, but this is not the fault of terminals or shells. It's really easy to create curses interfaces with libraries like blessings[1].

[0]: http://blog.z3bra.org/2014/01/images-in-terminal.html

[1]: https://github.com/erikrose/blessings

[theshetty]

I can't agree more!!!

I wrote this simple SAS program/job scheduler (https://github.com/PrajwalSD/runSAS) using just bash script (with few utils) intentionally to allow for maximum adoption in projects and just made it more more colorful. In short 75% of the code in there is just handling the terminal (keys, scroll, progress bar etc.) and 25% is the actual load balancing logic.

[thih9]

> I feel like we need a complete rewrite of terminal emulators/bash/whatever.

A complete rewrite would be an opportunity to overlook important ideas, compromise on goals and make design mistakes.

I’d strongly prefer slow, iterative approach here.

[kjeetgill]

To use a tortured metaphor: Like any pathfinding problem, sure slow and iterative is an important part of the algorithm but so is backtracking.

I'm inclined to "if it ain't broke don't fix it", but it's also just as true that this greedy algorithm just leaves us at a local optima.

[clon]

I am not looking forward to using a new improved shell written in JS.

[vkou]

I rarely look forward to using the old, crufty shell, but I'm a madman who would like things like arrow keys and backspace and maybe even box select to actually work in a sane manner.

I don't need spreadsheets and rich markup in my terminal. I just want basic HCI to work - consistently.

[hhas01]

The technical problem is you can’t effect worthwhile improvements in the terminal without also fixing the fundamental flaws below it: a huge mess of bloated, inconsistent, poorly-introspectable command executables, with only untagged, untyped “dumb” pipes to connect them together in the most laborious, unsafe way possible.

The logistical problem is, well… see all the other comments from those who don’t want it to change, whether through laziness, apathy, turf protection, or whatever. Next to the People Problem, the technical problem is the easy part to solve.

Incidentally, here’s what a good 1980s CLI looks like:

https://www.youtube.com/watch?v=o4-YnLpLgtk

Whereas the nix craphouse we’re now stuck with isn’t even a good 1970s* CLI.

Honestly, at this point I wouldn’t even bother trying incremental improvement. The *nix CLI does what it was built to do, and that’s all. When substantive change does come to text UIs it will be as revolution, not evolution.

Draw a line under it. Learn its lessons, both the DOs and DON’Ts. Then whiteboard from the ground-up what needs to come next. Work out where UI/UX needs to be in 2040 to effectively serve the billions of users of the generations to come, and make a start on building that. Dash ahead of history for a change, not be shackled and dragged back by it.

As old, traditional divisions between text, voice, and touch interaction become increasingly frustrating impediments in modern mass-market devices, there’s a killing to be made in smashing down all those old artificial barriers and replacing them all a single unified interaction model that can transition seamlessly between all three modes as its users’ needs and wishes direct.

[IshKebab]

I agree. Nushell (https://github.com/nushell/nushell) is the best I have seen so far. It still lives in a standard terminal from the 80s - basically no graphics, limited mouse support, etc. But I guess you can't change everything at once. It's an enormous improvement on Bash at least!

[WorldMaker]

I've wondered for a bit why in the Unix philosophy there isn't a standard /dev/progress or something for somewhat unified progress reporting already. It seems something that should be easy to standardize: make it append-only and take maybe a task name and a double [0, 1] (maybe, maybe optionally x of y counts, but a single double should be all you need). Let the terminal, the shell, and/or the GUI figure out how it wants to display things.

All of the major OSes today have some sort of dock/taskbar/activity panel standardized progress reporting API that is basically that simple, seems interesting we haven't pushed that idea back into the terminal. Makes you wonder what other sorts of GUI era pseudo-devices might be useful to add to Unix. In the other direction, imagine if some of these tools had been standardized as real devices in past terminals, like using something fun like an analog meter for progress operations.

[mixmastamyk]

> showing some progress bar, or resize the terminal window, it should handle it nicely like every other application.

I just added SIGWINCH (resize event) handling to my console/progress bar library, https://pypi.org/project/console/ , though it currently needs to be using the whole line. I will fix that in the future.

[nurpax]

Inline images would be great too (bitmap or svg). There’d be a lot less need for something like jupyter if you could show images in the shell.

[kjs3]

Why does a CLI need colours and icons? OK...colours have some value, but icons? Serious question.

[Shank]

On a basic level, a CLI with color adds visual information that's easily seen at a glance. In this example blog post, the author shows how adding color to `df` resulted in an immediate visual distinction between mostly empty and mostly occupied space, which would normally require scanning a column to find.

Icons? It's easier to show information in a smaller area when you can use the spot a character would take with an emoji or equivalent.

[kjs3]

I get that. So bold, italics and half-tone could be good enough for many/most use cases, and icons are just eye candy, right?

[enriquto]

Oh, please, no "color codes".

Some of us prefer a monochrome terminal, with the only distinction being bright/bold and regular text. I'm all for modernizing the classical unix tools, but turning them into kitsch Christmas trees is maybe not the best way.

[smabie]

You're telling me that you don't use colors with ls? This isn't even a matter of debate, ls with colors is superior and more efficient.

[basilisks]

`ls -F` is good enough for me