[comesee]

This is nearly what I'm talking about but it requires the terminal emulator to understand the image format. I guess that makes sense for transfer efficiency, and if you want a pixel-level API you could just transfer BMPs.

[daleroberts]

If you have iTerm2, you might like:

https://github.com/daleroberts/bv

https://github.com/daleroberts/itermplot