Listen to the pattern
A time-series is a pattern. It climbs upward and descends, patterns are stable or swinging, they change slowly or abruptly. The pattern is more important than the actual values. To understand the pattern it helps to read it aloud to yourself. “The first value is the smallest. A steep ascent follows. Then a descent roughly to the bottom again.” Time-series patterns are similar to sound patterns. A sound pattern climbs or descends just like values do. You can hear what you see and see what you hear. If you simultaneously hear and see the same your understanding is faster. Sounds read the pattern to you. Test audio sparklines.
Click on the large speaker symbol to play all patterns.
Click on a small speaker symbol to play a single pattern.
February 6th, 2007 at 8:03 pm
Bella,
I certainly can understand your delight in sound, but your claim that the addition of an audio encoding to a visual encoding of time-series values speeds up the process of understanding is not correct. It actually slows it down. I can see the pattern in an instant, because my brain processes the image in a parallel manner, but sounds are processed serially, which takes time. Audio encoding might be useful for people who are visually impaired, but it is not a good replacement nor even a good addition to a visual encoding in the form of a simple graph.
Stephen Few
Perceptual Edge
February 9th, 2007 at 11:51 pm
Bella
Time is a property of the auditory system, not of the visual system. As Bertin puts it:
“La perception sonore ne dispose que de deux variables sensibles: la variation des sons et le temps.
Par contre, la perception visuelle dispose de trois variables sensibles: la variation des taches et les deux dimensions du plan, et ceci hors du temps. Les systèmes destinés à l’œil sont d’abord spatiaux et atemporels. D’où leur propriété essentielle: dans un instant de perception, les systèmes linéaires ne nous communiquent qu’un seul son ou signe, tandis que les systèmes spatiaux, dont la graphique, nous communiquent dans le même instant les relations entre trois variables.”
Jacques Bertin, Semiologie Graphique
February 12th, 2007 at 12:23 am
Just another bit of information:
“Cognitive psychologists have provided myriad information about the effects of sound on memory and about the effects of multiple tasks on attention. From this research, auditory graph investigators have used two principles on the impacts of using sound and vision together for graphed data. One of these principles is that adding another modality (i.e., audition to vision) will result in redundancy in the information, and thus lead to better overall comprehension of graphs. The second principle is that having two modalities will result in divided attention leading to greater cognitive load, and thus to diminished comprehension of graphs. It would be useful for auditory graph researchers to determine in which situations these two contradictory effects manifest themselves. (…)
The results suggest that processing visual and auditory graphs together is a more difficult task than processing either alone.”
Terri L. Bonebright (2005): “A SUGGESTED AGENDA FOR AUDITORY GRAPH RESEARCH”
March 24th, 2007 at 10:22 am
Great idea.
I agree with Stephen Few that the pattern is grasped more quickly visually, but one should experiment with the length of the recording first. I believe that a trained ear can get the idea of time series dynamics more quickly than visually or at least get very close.
May 21st, 2007 at 7:09 pm
Would it be helpful to have a continuous tone playing quietly in the background representing the baseline/origin/zero of the y-axis? That might help convey which sounds (values) were at, above or below that baseline.
May 27th, 2007 at 4:19 am
[…] Bella consults » Blog-Archiv » Listen to the pattern Sound Spark (tags: sparkline) […]
May 29th, 2007 at 6:44 pm
I don’t find that the sounds enhance the ability to easily grasp the information. If one were to use this as an alternative for visually-impaired people, I agree with Scott Flodin that you need some way to indicate the baseline. Otherwise, you just hear low and high notes without any way to tell that they are above or below the baseline.
June 4th, 2007 at 2:29 pm
Great idea.
This might be useful for the visually impaired.
November 17th, 2007 at 3:18 am
I wonder if a continuos tone would work well. Maybe a flute or recorder.
Also any chance you would post the flash source?