[FloatArtifact]

I love seeing tools like this. I could see a light LLM for classifying elements augmented by Voice Recognition for Accessibility. Natural language will never be a great interface for High Domain, Low latency use case such as accessibility.

[casslin]

Thanks! Great idea-voice recognition is in our roadmap and exploring good open source options.

[FloatArtifact]

Quick clarification. Low domain knowledge is okay for those who don't have experience and don't know what to say, like Alexa. High domain is somebody who has expertise with a specialized workflow.

So, they will rely on voice commands for recognition, not natural language. Often one to two words to set a chain of tasks in motion. Think of having to control your entire computer, including navigating by voice. That would be very exhausting and inefficient through natural language. There needs to be a hybrid solution that can leverage low domain natural language, but also high domain command-based recognition. I cannot overstate how important of low latency between the beginning of a command and an action produced. High latency means a big cognitive load and not to mention just inefficiency.

There's a lot of overlap between UI automation and accessibility control tools. However, UIA automation has always been a slow process simply because the stack doesn't have the demand from devs for low latency.

It's a difference between having an independent agent do something on your behalf, not caring how long it takes, versus you waiting for a aynchronous task to be completed.

[casslin]

appreciate clarification. The low domain vs high domain distinction is spot-on:Latency kills expert workflows. keeping this in mind when integrating/designing voice recognition and more accessibility control options.