Hacker News new | ask | show | jobs
by kragen 753 days ago
i mostly agree. if you want to run an application that assumes ansi escape sequence support and a terminal emulator that doesn't implement ansi escape sequences, you can run a terminal emulator inside your terminal emulator, with the inner terminal emulator interpreting the ansi escape sequences and talking to the outer terminal emulator with whatever escape codes the outer one uses

three options for this inner terminal emulator are gnu screen (previously iscreen, which was a better name), tmux, and emacs term-mode. all three of these can use terminfo to figure out how to manipulate the outer terminal—but they wouldn't have to. mosh also does this terminal-emulator-inception thing but i think it's hardcoded to use ansi escape sequences to draw

— ⁂ —

what i'd really like to see is a reimagining of the terminal interface as something better than a vt340+. we have lots of historical things that could be sort of described that way (genera's lisp listener, the blit, mgr, caleb winston's stdg, notty, x11, postscript, html, react, wayland, opengl, vulkan, vnc) but they all fall short in one way or another. minimally i want a user interface that i can

1. write programs for in any programming language such that

2. simple programs like 'hello, world' or displaying a two-column table of numbers are about as simple as in an xterm, and

3. the programs' input and output is manipulable by the user at various levels of abstraction (copy/paste, screenshots, editing the output text, automating interactive workflows (like expect/autohotkey/selenium), screen readers, asciinema/screencasting, livestreaming, obs studio), and

4. the program is interactive (html kind of fails at this), and

5. the user interface can be used on a separate computer from the program (maybe the user interface runs on a cellphone while the program runs on a rack in a data center, or maybe the user interface runs on a laptop while the program runs on a quadcopter), and

6. small computers (under a megabyte of ram) can be used to show the program's user interface, but

7. the program can display anything the user's monitor is physically capable of displaying and support any of the user's input devices, including mice, touchscreens, accelerometers, cameras, etc.

emulating a vt340+ like xterm does gives you the first six of these, which is a big reason why people keep writing new programs for that platform, but totally flunks the last one; you can't even display text in a proportional font, much less a photo or a video

obviously there's some degree of tradeoff in some cases between 5, 6, and 7, because you aren't going to stream 4k video over a 9600-baud serial line, and you aren't going to decode it on a 286 (nor are you going to stream webcam video back from the user if the webcam is plugged into a 286), but there's a pretty broad spectrum today of {comms bandwidth, terminal cpu, desired output} triples for which performance isn't a hard problem—all you have to do is not totally fuck it up like sixel

i don't know, i'm not waiting around, i'm gonna be over here hacking on godot instead

1 comments

> emulating a vt340+ like xterm does gives you the first six of these, which is a big reason why people keep writing new programs for that platform, but totally flunks the last one; you can't even display text in a proportional font, much less a photo or a video

xterm supports sixel graphics.

"sixel" is an image file format; the ImageMagic "convert" command can convert other formats to it. Just cat a sixel file to stdout in an xterm and it's displayed in the window.

The implementation is not 100% solid (for example, there's a size limit), but it does work. And tmux can be build with sixel support, so you can display sixel images in tmux under xterm.

    convert foo.png sixel:-
or

    convert -geometry 600x600 large.png sixel:-
https://en.wikipedia.org/wiki/Sixel

No, there's no video support (unless I've missed something).

iterm's protocol can display gifs, could use a streaming method via that, Kitty's protocol supports manual animating, though I have a feel it wouldn't be able to do video.

mpv supports both sixel and kitty's method for doing video

Based on my (limited) understanding of Kitty's image protocol[0] (a.k.a the "Terminal graphics protocol") video support could be potentially implemented via multiple approaches, including at least:

(1) Draw image repeatedly using the "don't move cursor" cursor movement policy[1a][1b].

(2) Draw image replacing an existing image with the same id.

(3) Use animation configured to use "loading mode"[2] ("in this mode when reaching the last frame, instead of looping, the terminal will wait for the arrival of more frames").

An actual terminal graphics/image protocol enables some IMO[3] pretty cool possibilities:

* A while back I implemented a proof-of-concept that displayed a simple interactive egui (a Rust immediate mode GUI project) GUI within WezTerm via the protocol.

* Later I discovered someone had gone as far as making a port[5] of the OpenGL gears demo to kitty terminal graphics protocol.

* And there's even a "full graphical web browser for Kitty terminal with mouse and keyboard support"[6][7].

---- footnotes ----

(Content advisory for links: Kitty terminal graphics protocol spec includes a sample image whose subject has requested that the image no longer be used for such purposes.)

[0] https://sw.kovidgoyal.net/kitty/graphics-protocol/#terminal-...

[1a] https://sw.kovidgoyal.net/kitty/graphics-protocol/#controlli...

[1b] This approach appears to be the method used by the mpv implementation: https://github.com/mpv-player/mpv/commit/874e28f4a41a916bb56...

[2] https://sw.kovidgoyal.net/kitty/graphics-protocol/#controlli...

[3] I'm aware that not everyone will think static or dynamic image display is something a terminal "should do" but to me it seems better to have actual image support in terminals rather than use (already currently used) existing "hacks" that provide a poor facsimile of support[4].

[4] Then again, I'm also of the opinion[4a] that perhaps terminals should support actual graphical UIs rather than the poor facsimile of them delivered by TUIs. *cough* :D

[4a] Motivated by the thought that at the current point in time perhaps the "essence" of a "terminal" is its linear "chronological" presentation of input/interaction/output history rather than its use of "text". (See also: "lab notebooks" & some tiling window manager features.)

[5] https://github.com/michaeljclark/glkitty

[6] https://github.com/chase/awrit

[7] Which I imagine DEC is turning in its grave about. :)

i think the kitty protocol is a great deal less detestable than sixel ;)

it's unfortunate that it's still only six bits per byte, but at least it supports png and 24-bit color