[tim1994]

Assuming this does not yet exist: If anyone is interested in building something like this (that works offline) please let me know, I'd like to join. I have some experience building web extensions but unfortunately not the time to develop & maintain this on my own right now.

[user3939382]

All you need is a CSS selector for the aria label and a click action. Once the voice transcription is done it’s 1 line of JavaScript. You’d have to rely on the aria labels though.

[tim1994]

It's not quite that simple. For example the button doesn't necessarily have to have an aria label. Could be the (nested!) content. There could be multiple candidates (buttons). The speech recognition thing is also not trivial. Yes there are WebAssembly based solutions but those have to be properly integrated and tested (not sure how good they actually are)

[bryanrasmussen]

for a Cucumber based testing solution back in 2013 the company I was working for had all element interaction commands go to a function that tried to identify what element it could be so you could write "I click 'home'" and that function would do

1. look in a object we had in which a bunch of DOM objects have been identified via CSS selectors of XPaths to see if there was an object named 'home' if that not exist

2. look for an element with the id home if that not exist

3. look for an element with the class home - take the first one. If that not exist

4. look for an element in which the text node was 'home'

There were a few other heuristics to determine what to do also dependent on if you said I select home, I click home, I move the mouse over home etc.

[Jemm]

Same.

[loa_in_]

Neither of you have a way (in your profile) to contact you outside of the website

[tim1994]

Perhaps a caching or timing issue? I updated my profile shortly after posting this (and noticing it myself).