Hacker News new | ask | show | jobs
by imglorp 1002 days ago
I was sighted volunteer on a call with a fellow using a treadmill touchscreen. He already knew the menu flow but the UX was dynamic and it wasn't his screen or he could put locator dots on it (Lesson to all designers! Hardware can have physical buttons!). Our interaction was mostly him stating his goal determining the screen's starting state and then where UI elements were, and I would feed back his finger position like "a little left ... no, too far, now up a little ... ok hit it."

I think we can imagine an AI could describe the screen, and even find non-language visual elements if asked explicitly, like arrows or turtle vs hare icons etc. But is it ready to have shared context of how people need to interact with that UI?

2 comments

> But is it ready to have shared context of how people need to interact with that UI?

And what if it is not? Does the AI have to be useful for every single possible use case before it can be used?

You are reacting as if they are proposing to remove the already existing venues of help. When I see no sign of that.

Ehh, give it a few years.