[user-the-name]

Imagine if we had terminals that could handle graphical data. We wouldn't have to do weird kludges like this, we could just plot the waveforms in the output of our tools.

But it's 2021, and not only is this not possible, there is not even a path forward to a world where this would be possible. It's just not an option. Nobody is working on this, nobody is trying to make this happen. We're just sitting here with our text terminals, and we can't even for a second imagine that there could be anything else.

It's sad, is what it is.

[charlesdaniels]

I would point out that sixels[0] exist. There is a nice library, libsixel[1] for working with it, which includes bindings into many languages. If the author of sixel-tmux[2][3] is to be believed[4], the relative lack of adoption is a result of unwillingness on the part of maintainers of some popular open source terminal libraries to implement sixel support.

I can't comment on that directly, but I will say, it's pretty damn cool to see GnuPlot generating output right into one's terminal. lsix[5] is also pretty handy as well.

But yeah, I agree, I'm not a fan of all the work that has gone into "terminal graphics" that are based on unicode. It's a dead-end, as was clear to DEC even back in '87 (and that's setting aside that the VT220[6] had it's own drawing capabilities, though they were more limited). Maybe sixel isn't the best possible way of handling this, but it does have the benefit of 34 years of backwards-compatibility, and with the right software, you can already use it _now_.

0 - https://en.wikipedia.org/wiki/Sixel

1 - https://saitoha.github.io/libsixel/

2 - https://github.com/csdvrx/sixel-tmux

3 - https://news.ycombinator.com/item?id=28756701

4 - https://github.com/csdvrx/sixel-tmux/blob/main/RANTS.md

5 - https://github.com/hackerb9/lsix

6 - https://en.wikipedia.org/wiki/VT220

[csdvrx]

> If the author of sixel-tmux[2][3] is to be believed[4], the relative lack of adoption is a result of unwillingness on the part of maintainers of some popular open source terminal libraries to implement sixel support.

If you have any doubt, look no further than this thread: the sixel format is attacked not for any technical reasons, but for its age, RIGHT HERE ON HN:

>> "That's a protocol that's a good forty years old, and even that is not supported. And I can see why, why on earth would you want to be adding support for that in 2021? What a ridiculous state of affairs."

What's ridiculous is, with so many examples and quotes, some people still thing I must be "emotional" (I had a long discussion here... https://news.ycombinator.com/item?id=28761043 ) or that a few million colors is not sufficient for the terminal (!)

There is none so blind as those who will not see...

[db48x]

Sixels are fun, but I was disappointed by libsixel. It’s not really a general–purpose library; most of it is there only to implement various command–line arguments of img2sixel. Most of the functions determine what to do by parsing strings taken from the command–line arguments, so reusing it is super annoying.

When implementing a program that outputs sixels, you are better off looking elsewhere. SDL1.2-SIXEL is a good choice in general, if you are writing C or don’t mind using the C bindings for your preferred language.

[jwosty]

That’s interesting. Do you think sixels could work for the baseline tests? Would it be feasible to have them display nicely in an IDE, like VS Code or Visual Studio?

[charlesdaniels]

I don’t see why sixels couldn’t work. You’d probably want a tool to decode them, diff the images, and then output another sixel image. I’m admittedly not sure of such a tool existing off the shelf though.

I’m not aware of text editors supporting sixels, which could make preparing the tests a challenge. Certainly, you could imagine a text editor supporting them, but I’m not aware of one that does personally.

I will concede that for your specific use case, an off the shelf ASCII plotting library probably involves less custom tooling.

[csdvrx]

> I don’t see why sixels couldn’t work.

Sixels will work: they are fast enough to allow youtube video playback !!!

https://github.com/saitoha/FFmpeg-SIXEL/blob/sixel/README.md

The problem is NOT THE FORMAT, the problem is the lack of tooling: links and w3m are among the rare text browsers that can display images in the console.

It's just a matter of the browser sending the image to the terminal in some format it can understand, but if that hasn't be thought about as a possibility (say, for text reflow issues) it's going to be far more complicated than just adding a new format, as you will have to work both on say the text reflow issues (ex: how do you select the size of the placeholder, when expressed in characters?), on top of the picture display issues.

Said differently, it would be easier to have console IDE that supported graphics if any format whatsoever (sixel, kitty...) was supported by a console IDE; we could then argue about the ideal format.

Arguing about the ideal format BEFORE letting the ecosystem grow using whatever solution there is only results in a negative loop.

It's like if a startup was arguing about the ideal technological stack even before trying to find a product market fit!!

Personally, I do not care much about sixels, kitty or iterm format - all I want is to see some kind of support for a format that's popular enough for tools using it to emerge.

Yes, it would be better if that supported format was the option that had the greatest chance of succeeding, but right now, that is a very remote concern: first we need tools, then if in the worst case they are for a "bad" format, we can write transcoders to whatever format people prefer!

Right now, there is rarely any "input" to transcode (how many console tools support say iTerm format?), so we have a much bigger bootstrapping problem.

> an off the shelf ASCII plotting library probably involves less custom tooling

With a terminal like msys2 or xterm, no custom tooling is required: just use the regular gnuplot after doing the export for the desired resolution, font, and font size.

gnuplot is far more standard than plotting library that often require special Unicode fonts on top of requiring you to use their specific format.

[MayeulC]

I find kitty's graphics protocol to be a superior implementation of the idea: https://sw.kovidgoyal.net/kitty/graphics-protocol/

[user-the-name]

That's a protocol that's a good forty years old, and even that is not supported. And I can see why, why on earth would you want to be adding support for that in 2021? What a ridiculous state of affairs.

[charlesdaniels]

> That's a protocol that's a good forty years old

34 years old, actually. I guess we can go ahead and deprecate the x86 instruction set, tcp/ip, ASCII, C, tar, and many other tools and standards that are old.

> and even that is not supported.

xterm supports vt430 emulation. I use this semi regularly. I believe mintty also supports sixels, plus a handful of others. The libsixel website has a full list.

> And I can see why, why on earth would you want to be adding support for that in 2021?

You might want to read your own post ( https://news.ycombinator.com/item?id=28856005 ).

What’s your great idea as opposed to sixels?

[gwbas1c]

> It's sad, is what it is.

With graphics being everywhere in 2021, I wouldn't call this situation "sad," I'd think a lot more critically about why.

To start with, fixed-width text is significantly easier to work with than graphics.

Nothing's stopping anyone from writing a CI tool that outputs to HTML with embedded images. The bigger question is why it's uncommon.

[MayeulC]

In truth, it's because text is quite easy to handle. It's easy to make a program that handles text, too.

And so we have a lot of text editors, diff tools, efficient compression, tools like sort and uniq: the whole unix ecosystem.

So if you transform sound to text, you can then use text tools to compare the output to catch differences. A simple serialization of numerical sample values would have caught the bug, but I agree that having a way of visualizing the output is nice.

Command line input, programming, etc. is also still mostly done with text, because it's easy to transform. Of course, you can imagine working at a higher level with objects (like powershell does IIRC), mimetypes, etc.

[rbanffy]

The venerable xterm and a lot of later physical terminals (those things with CRTs) can emulate Tektronix (Tektronix, that today makes instruments, also made computer terminals with fancy storage CRTs that were kind of e-paper-like, but green - and sometimes yellow - screen) graphics. iTerm2 and some others, as pointed out, can do Sixel graphics (a format designed originally for DEC dot-matrix printers that some DEC terminals also implement).

[user-the-name]

I mean, yes, that is how sad the current state is.

[rbanffy]

VTE and, with it, almost every Linux distro, will get Sixel support soon. I volunteered to add Tektronix graphics to it too, but this is neither a dire need, nor something I have done before, so it'll take some time.

[user-the-name]

It's forty years old. Why on earth would you be adding that in 2021?

Why are we not focusing our energy on making something that is actually up to date?

[rbanffy]

Because things that existed 40 years ago are useful, already have software written for it, are compatible in sometimes unforeseen ways (a DEC dot-matrix graph can be printed as is on a Sixel-compatible terminal!) and have been battle tested for ages.

There is a reason the Unix way of bytestream-based shell and pipes is still useful and present these days to the point that That Other OS is now embedding Linux in it.

Also, these ancient terminals often had some interesting typography options that are encoded in the ANSI standard that most modern terminals don't bother (line attributes that generate wider and taller cells are one such example).

These formats may be more desirable than more modern and complete ones such as PostScript for other reasons. I wouldn't advise implementing a terminal capable of rendering PostScript graphics because it's one more way to infiltrate malware in your computer by rendering untrusted inputs (There are a lot of RCE opportunities in exploiting vulnerable decoders).

[voldacar]

In TempleOS you can mix text, images, hyperlinks, and 3d models in the terminal. This is true for the whole system: you could literally have a spinning 3d model of a tank as a comment in a source file. That's right, it took a literal schizophrenic to make an OS with a feature that should have been standard decades ago.

Nobody tries to make actually interesting new operating systems anymore. OS research today is just "let's implement unix with $security_feature", nobody is actually trying to make computers more powerful or fun to use, or design a system based off of a first-principles understanding of what a computer should be.

God I wish I was born in the lisp machine timeline

[PeterisP]

The features you describe belong to the app ecosystem, not to the OS - IMHO the OS is about hardware and drivers, and what kind of graphics is supported by your terminal and source file editor is orthogonal to the OS and could be done in any of the current OS'es; but that would require a rewrite/redesign/reimagining of the whole standard application package which seems a much larger project than "merely" an OS.

[voldacar]

"There are more things in heaven and earth, Horatio, than are dreamt of in your philosophy"

An OS facilitates communication between programs running on a computer. Unix lets those programs communicate by sending characters of text to each other. You could just as easily imagine an OS that lets them communicate by sending images, audio/video, 3d models, etc. An OS can be way more than what you think it is. To detox your brain from this unix worldview, spend some time in a VM and play around with amigaOS or opengenera. Those were actual coherent OSes with an actual view of what a computer should be and how it should behave. Unix isn't.

> reimagining of the whole standard application package which seems a much larger project than "merely" an OS.

By OS, I don't mean kernel. I mean the base set of software that lets you interact with your computer and do interesting stuff with it.

[rbanffy]

The line between app platform and OS is a blurry one. The Amiga OS, for instance, has libraries for specific file types that expose standardized entry points. This way, if you install the library for Photoshop files, all graphics programs that adhere to that protocol will be able to read and write Photoshop PSD files. Microsoft had DDE and, later, OLE, for embedding objects from one program into data from another in a standard way all programs were supposed to share. It was a pain.

This blurry line is present in other environments as well. In the Apple Lisa, installing a program resulted in new templates in the Stationery folder. In Smalltalk, installing a program adds its class definitions to the system as independent entities you could use in your own programs.

Not all operating systems are the children of Unix and VMS.

[bitwize]

Smalltalk's components were so tightly interdependent that their integration smoke test was 4+3, because evaluating a simple addition expression exercised like 3/4 of the entire system.

[rbanffy]

I remember once crashing Squeak by doing

`True := False`

Should crash any vintage ST/80 workstation.

[woodrowbarlow]

the downside of rich terminal output is that media formats become the system's responsibility. applications can't output media in formats that aren't provided by the system, because then the terminal wouldn't know how to display it and interop with other applications (e.g. piping) wouldn't work either.

[voldacar]

You could let a program create an API for manipulating a new type of data and inform the system about it so that other programs could use it. This is more or less what AmigaOS did; you installed a datatype for e.g. a PSD file, then all your programs that worked with images could read PSD files. I think it's a nice idea.

[HPsquared]

Notebook interfaces are basically that, e.g. Jupyter or Mathematica.

[kevin_thibedeau]

This isn't a graphical problem. All that's required is storing arrays of validation data and a diff tool to check for mismatches. Visualizing the results is useful for failure analysis but not a core requirement. That can readily be done with free tools like matplotlib. We live in that world today.

[outworlder]

Some terminals can.

https://iterm2.com/documentation-images.html

That's iterm's own implementation. There's also sixel, as pointed out by another comment.

[zokier]

> Imagine if we had terminals that could handle graphical data.

We have. They are called "browsers". You might be even using one right now!

[thanatos519]

Maybe you would like to support https://ctx.graphics/