[ansible]

Here are my comments and concerns.

1. Minimal latency from keyboard to screen.

2. As others have mentioned here, awareness of line wrap, so that cut and paste work correctly at all times. Not just for raw output from a command. Editors too should communicate with the terminal to indicate that it is wrapping lines.

3. 24-bit color.

4. Apps (shells, editors) can query and set the window icon as well as the window title.

5. Integrated graphics display. So that I can just 'cat' a picture to display it in the terminal window, instead of using an external application.

[jstimpfle]

"cat" could never work of course, or the general idea of a terminal that has only a pair of input/output streams must be given up. I don't think people are willing to do that.

A sensible protocol to draw raster graphics would be nice, though. (And it might exist, I don't even know). I don't care whether I need to type "cat" or the name of a terminal-based image viewer.

[terminal-survey]

> "cat" could never work of course

We have an idea in that direction, where each program in a pipeline can optionally specify a file type, and the last file type hint in the pipeline determines how the stdout is rendered. So if you have "cat image.jpg", then "cat" can indicate a file type of image/jpg, causing stdout to be interpreted as an image. But if you have "cat image.jpg | grep foo" (just making something up here), then grep can indicate a file type of application/octet-stream because it cannot guarantee what the file type is.

The tricky part, from what I can see, is explaining the pipeline topology to the terminal, so that it knows that "grep" comes after "cat", even though command parsing happens in the shell only. That's still a big unknown. A previous project in this area (TermKit) mandated that applications specify a "Content-Type" header (like in HTTP) in their stdout, but that would break most existing programs, so it's not an option for us.

[jstimpfle]

I would not recommend that. That would be the source for endless bikeshedding and could never satisfy all people. (See the mess that HTTP / REST are).

It's not a problem to have this handled in the shell such that the output is rendered in such a way as to please the terminal. You can do this manually (e.g. append a "| display-jpeg-in-terminal" command to the pipeline), or use a magic bytes based file viewer.

Look at "run-mailcap" or its shortcuts like "see". It does basically that, and can also be fed through standard input (which covers the piping situation).

[baq]

if shells had some more types than a stream of bytes, it'd make sense for 'image' type data to be displayed as images. the pipes you mention are then simply type casts or conversion functions.

[jstimpfle]

I don't know what you think is missing. "Displaying 'image' type data as images" totally works. Are you aware of magic bytes (which I mentioned above)?

[baq]

never used them, but i'm thinking more in terms of powershell (pretty sure displaing images isn't something it does though). if images in particular work ok then that's one thing that i can scratch off my list of annoyances with ssh.

[jitl]

There are already terminals and cat implementations that can display 24-bit images just fine in the terminal, by using special escape codes that each encode a pixel. No changes to the IO model needed.

https://github.com/saitoha/PySixel/blob/master/README.rst

[jstimpfle]

Sure - this is what I mean by "protocol". In the case of the image format you linked even cat does work, because the image format contains pre-rendered escape sequences. This does not work in general (e.g. for JPEG or other formats) - where you need a program that does the transformation into escape sequences.

[lornemalvo]

While plain `cat` does not work, using escape sequences and providing support in the terminal emulator allows already to display images inline. See the iTerm2 implementation here: https://www.iterm2.com/documentation-images.html

Making plain `cat` work too should not be too big of a problem with some small support from the running shell. iTerm2 also knows when a command is starting to run/exits, parsing the output it retrieved from STDOUT in between by matching it against some magic numbers and displaying the content accordingly is not too difficult.

[jstimpfle]

Then it's not cat anymore. The purpose of cat is to output the inputs unchanged and concatenated. If you need magic, that's a task for another program.

[lornemalvo]

`cat` is doing exactly that. It is up to another program, namely the terminal emulator to interpret its output.

[jstimpfle]

The terminal emulator does not (and should not, according to my strong opinion) interpret the output of different programs differently.

In fact it's not currently technically possible for a terminal emulator to know which program is writing into it. It maybe many programs simulatenously, or it may be the kernel asynchronously, etc... If you want to change that you would have to make the architecture tremendously more complicated and less flexible.

And the same goes for shells. You don't want to complicate the architecture just so that a lawyer could attest you that "it was cat" who drew the image und some weird interpretation. Nothing would be gained, and nobody would be able to understand what's happening anymore.

[Sylos]

> 5. Integrated graphics display. So that I can just 'cat' a picture to display it in the terminal window, instead of using an external application.

Enlightenment's Terminology has this. It's a separate command "tycat", but yeah, it can display images and videos in the terminal.

They also have "tyls", which will create little thumbnails when listing picture or video files and then you can click on the thumbnails to show the full thing.

https://www.enlightenment.org/about-terminology

[lscotte]

So basically you just want an Xterminal. This is pretty much exactly why X exists...