[sitkack]

What if you had a OCR overlay that had a DOM like structure along with coordinates? You could hit a key combo, and then the text dom could float over the page and you could select nodes of it using your keyboard modifiers of choice (vim, emacs, etc).

You could also read the process memory directly by a supervisory program, like a debugger, to extract or insert text directly. It would be like idea implantation.

Does that make sense?

[bitofhope]

OCR hardly needed. Just read /dev/text.

I don't think the mouse chording is that essential to Plan 9. It's how rio and acme work, but those programs can be replaced without throwing away most of the OS concepts. I think you could absolutely create a Plan 9 UI that, for example, uses vi-like key bindings to jump around windows and text. Nobody's just done it yet and hardcore users are happy enough with what they have for now.

[arexxbifs]

I think it'd be easier to just implement better keyboard support in E.G. Acme and, to a lesser extent, the terminal. :)

Side note: Inserting text using a debugger shouldn't be necessary. Everything really is a file, including Acme's windows and their contents (which is pretty cool).

[sitkack]

Ok, what if all the programs had an file, like a port, where it could read and write messages encoded in a high level dynamic language, maybe like Lua, or Arexx :) ?

Then you could send it a message

    GET /text_repr?encoding=json

[ta988]

Durée has something like that http://durden.arcan-fe.com/

[sitkack]

This whole stack blows my mind. What kind of wizardry is this?

[ta988]

I know! They have a shell that is pretty insane too.