[got2surf]

I've done about 4 years of research on sonification, which is using non-speech audio to represent patterns in data (http://mags.acm.org/interactions/20120102/?pg=37#pg37 for some specifics). This article is a subset of sonification in some ways, since we're representing some quantitative data using auditory parameters.

There's an entire class of scenarios where conventional HCIs can't represent data for analysis: where people have an occupied visual sense (doctors during surgery), where people are mobile, where people are overloaded by visual data (stock analysts), where the visual sense isn't suited for extracting data from noise (during the Voyager 2 mission), etc. We tend to rely only on our visual sense for communicating data, and as we start using computers for data display in more places, we're reaching the limitations of conventional HCI.

My research was on proving the viability of sonification - looking at the accuracy of comprehension, the cognitive and physiological processes, demonstrating shared mental processes with visual graph comprehension, etc. It's still something I'd love to revisit and commercialize someday.

[specialist]

No doubt.

Years ago, a buddy hosted corporate email, dns, etc. Racks of servers. He added ambient audio to everything he monitored. Nature sounds, weather, birds, insects, etc. The volume, samples, and tempo would change dynamically. Happy soothing sounds when all was well. Disruptive sounds when bad stuff happened.

(I don't know if you'd classify that as sonification.)

Walking around, visiting with guests, talking on the phone, his crew always knew the health of their systems.

It was awesome.

[got2surf]

Yep, exactly - applications like that are just cool, useful and intuitive all at once.

What's interesting is that for visual displays, we have decades of detailed research into visual perception - we know from experience how to design a graph so that patterns can be quickly extracted and understood. We don't yet have that same level of understanding for sonifications, but once we do, applications like this will be even better

[ehsanu1]

Very interesting. Related: this colorblind man hooked up a camera that processes color into an audio signal which he can hear and developed a sense for color: http://www.ted.com/talks/neil_harbisson_i_listen_to_color.ht...

[dangrover]

My old piano teacher actually set up trading desks using sonification technology: http://www.icad.org/websiteV2.0/Conferences/ICAD2004/papers/...