Hacker News new | ask | show | jobs
by milansuk 1544 days ago
Do operation systems have APIs, so the program can send them what is on the screen? I mean just description like: There is the button with title "Push me" in the rectangle(50, 50, 100, 70), etc.

Or are there any "screen reader" libraries?
6 comments
malkia 1544 days ago
Even systems like Qt, where things are drawn using non-native widgets support that (I was long proponent of wxWidgets, because I thought assistive tech is only support only if you use native widgets. The idea of API where you explain what you are showing, and how to interact with it never came to my mind, but with ARIA - like any browser that's the way to go).

Not only that, but I thhink assistive tech should allow you do to easier GUI testing too, but haven't digged into that idea yet (probably folks already are)

link
jcelerier 1544 days ago
Yes, OSes have APIs. Another commenter talked about the Mac one, Linux has AT-SPI and Windows has MSAA (historic) and UIA. If I'm not mistaken, they generally are about representing the UI as a tree of items as this is what the accessibility hardware expects / works best with.
link
a-priori 1544 days ago
Yes. I don’t know about Windows but I know that for OSX you need to implement the NSAccessibility protocol in AppKit. This gives you a way to describe the hierarchy of the UI and to notify the system of important changes.
link
fatih-erikli 1544 days ago
There's a10n standards defined by w3c, https://www.w3.org/TR/using-aria/.
link
pjmlp 1544 days ago
Yes they do, at least Apple, Google and Microsoft ones.

On GNU/Linux, I think the only thing that exists in the Gtk+ features that were originally sponsored by Sun.

link
underwater 1544 days ago
Aren't you then trying to bridge an immediate mode API (rendering controls) and a retained mode API (assistive tech).
link
mwcampbell 1544 days ago
Yes. This has influenced the design of my AccessKit [1] project. With AccessKit, the application (or GUI toolkit) pushes tree updates to the platform adapter, which maintains a complete tree that it uses to implement the platform-specific accessibility API. Each application-supplied tree update can be a full tree or just the nodes that have changed. So an immediate-mode GUI can push a full tree every frame. There just has to be some way of keeping node IDs stable across frames.

[1]: https://github.com/AccessKit/accesskit

link