[kingsuper20]

I spent an hour or so messing with Siri on a Mac Mini I have lying around from an old contract gig. Wondering how I would use it without much (or any) vision. It's directly hooked to a fairly fast lump of hardware and communicating with a potentially huge back-end.

Utterly maddening, barely useful, overly tied to Apple products, and it's a design problem I had never considered before. Kind of disappointing after watching some video of Tesla auto pilot do it's thing, perhaps natural language processing is harder.

Other people have gone down this road, but it would be interesting to think about how a purely audio and button based internet and phone system would act. It's a wonderful thought problem. It could well be that the solution is not in website redesign, but in the parsing and analysis of the website. How would you describe a banking site to a sightless person?

[jodrellblank]

My understanding is that blind or visually impaired people tend to use Windows because that's where the two(?) big screenreader programs run; JAWS is one[1], I forget the other but it might be NVDA[2]. Microsoft have traditionally had some focus on accessibility, e.g. there's a video from a few years back of a blind programmer demonstrating how they program in Visual Studio[3]. Outside that, Emacspeak[4] is an eyes-free Emacs, I think it can boot into that environment; it was developed by a blind person and claims:

> Emacspeak introduces several improvements and innovations when compared with screenreaders designed to allow blind users to interact with personal computers. Unlike screenreaders that speak the contents of a visual display, Emacspeak speaks the underlying information. As an example, using a calendar application with a screenreader results in the blind user hearing a sequence of meaningless numbers; In contrast, Emacspeak speaks the relevant date in an easy to comprehend manner.

> The system deploys the innovative technique of audio formatting to increase the band-width of aural communication; changes in voice characteristic and inflection combined with appropriate use of non-speech auditory icons are used throughout the user interface to create the equivalent of spatial layout, fonts, and graphical icons so important in the visual interface. This provides rich contextual feedback and shifts some of the burden of listening from the cognitive to the perceptual domain.

[1] https://www.freedomscientific.com/products/software/jaws/

[2] https://www.nvaccess.org/download/

[3] https://www.youtube.com/watch?v=94swlF55tVc

[4] http://emacspeak.sourceforge.net/

[kingsuper20]

I'll break out a Win10 PC and give it a shot, although it seems like ipads and iphones get most of the accessibility credit. I was also fooling around with those sorts of features on a Chromebook.

I can't help but wonder if simply reading the screen is the wrong answer. It's as if the important thing is to understand the screen and then explain it without the need for any visual/location kind of concept. Another angle would be to write a completely audio-based alternative interface for important websites.

No doubt people a lot smarter than me have spent a lot of time on these issues, but I'm pretty dissatisfied with what I've seen so far.

[dervjd]

Siri isn't the same thing as Voice Over/Voice Control! [0]

Apple has been frequently recognized for their efforts to make their products accessible - including by the National Federation of the Blind (NFB) [1]. Not to come off as a shill, but their contributions in the area of accessibility are worth praising.

iOS was the first mobile OS to include accessibility features. Apparently they're now using the LiDAR scanner on the iPhone 12 Pro to power a "people detection" feature for people with sight impairments.[2]

Here's a few of Apple's marketing videos on their accessibility features that are kind of neat:

A blind drummer who uses an iPhone with a black screen: https://www.youtube.com/watch?v=EHAO_kj0qcA

Accessibility: https://www.youtube.com/watch?v=XB4cjbYywqg

Voice Control: https://www.youtube.com/watch?v=aqoXFCCTfm4

---

[0] https://support.apple.com/accessibility/mac

[1] https://www.afb.org/aw/16/6/15452

[2] https://support.apple.com/guide/iphone/people-detection-iph4...

[kingsuper20]

I've used Voice Over to some extent also (once again, this isn't an exhaustive survey by any means). It does appear to be oriented to people with motor issues more than the blind.

As I've said a couple of times here (mea culpa), my gut feeling is that the philosophy is wrong. By building verbal control systems on top of (over-) visually dense user interfaces, you have to run just to stay in place.

It could be that the answer is an entirely different audio UI for computer management and then the replacement of standard applications with ones that are suitable for audio control (with perhaps a few buttons). Perhaps somebody has made an email app for blind people, but I haven't run into it yet. It certainly wouldn't be Outlook with a robot reading the screen to you. Probably the only way I could understand this at all is to attempt to produce one, writing software has the side effect of forcing you to fully understand a problem.