| > you have to have the right information shown in the right ways for people to develop a correlative, intuitive understanding of what they've built Lots of things have to be right for this to work, unfortunately, and company dashboards I've seen so far tend to be nowhere near it. For instance, the dashboard refreshed $PERIOD only makes sense if you're showing data that updates $PERIOD, and if you can respond to changes in that data $PERIOD. $PERIOD = "in realtime" or "every minute" or "hourly" or whatever is relevant in a given context. If you're looking at the dashboard much more frequently than the data changes, you're wasting time. If the data changes much more frequently than you're looking at it, you're likely to miss things, as 'geofft mentions elsewhere in the thread. And if you can't react to the data roughly as fast as it's updating, there's no point in looking at it so often. All those periods - recording, observing and reacting - must be roughly similar for the always-on dashboard to be useful, relative to generating reports every now and then. Panels full of lights and charts work on fighter jets or on the bridge of the Enterprise, because the pilots/crew are in a tight feedback control loop with their dashboards. (WRT. reacting in time, there are also error bars to consider. For instance, people on a diet are advised to weigh themselves weekly and not daily, because body mass varies by +/- 2kg during the day, so a naïve person checking weight daily would get fixated on those random oscillations. It's easier to tell regular people to reduce measurement frequency than to explain to them what a low-pass filter is and how is it relevant here. I have a feeling there's plenty of dashboard misuse that amounts to that too.) -- Speaking of the Enterprise and "getting the feel for the system", there's something that I'd like to try one day: make a monitoring tool that translates various system metrics into background sounds, creating an ambience similar to the one you hear on the Enterprise-D[0][1]. I feel a somewhat unobtrusive mix of background noises would be better to develop "the feel for the system" than a visual dashboard. Real-life examples of this are combustion engine's RPM, or spinning rust hard drives, if anyone still remembers those. -- [0] - https://www.youtube.com/watch?v=UKBvaOLDem0 - the bridge [1] - In Enterprise's engineering, there's a well-known pulsating sound of the warp core; I can't find a good enough YouTube video (whatever there is, apparently got broken by YT's audio compression). This background pulsing correlated to the speed Enterprise was traveling with. |
For me, good dashboards are like good checklists: they should be living entities owned by the team in question and regularly updated to address active concerns. And they don't even have to be complex. Back before CI was in fashion, I drove giant changes in a team's behavior just by having a single LED indicator (the now-departed Ambient Orb) show the state of the current build. Previously, the build would stay broken for weeks at a time, only converging to green around the time of release. Nobody liked that, but they were used to it, so they'd just work around it. But once it was visible and discussed, they eventually got so the build was green almost all the time. It was less painful and saved a bunch of time.
I would absolutely love to try out a set of ambient audio indicators. I suspect I'd want to try it along with a visual dashboard, because the moment I hear something anomalous, I'm going to want to look up and see the recent history, so I can correlate the audio with what it represents and what else is going on.