Sonifiying searching through a data set.

Camille Goudeseune

Searching through a dataset.

Moving a probe or a cursor through a dataset can be sonified in two ways, continuous or triggered. You can play back a single triggered sound (swoops and beeps) when the user clicks a wand button, or play back a sequence of them while the user holds a wand button. You can also play a continuous sound that starts when the user depresses a wand button and ends when they release it.

If the dataset is scalar, choosing a range of sounds is fairly straightforward. Use audpanel (see below) to find two slider-settings which will correspond to a minimum and maximum value of the scalar. Where your code has the scalar's value, just use linear arithmetic to fill the array to be passed to AUDupdateSimple() so the array has the first slider-setting when the scalar is at its minimum value, and the second slider-setting for the maximum. The file exampleScalarProbe.c does this.

If the dataset is vector-valued, choosing an effective family of sounds is harder. This is because the mapping from probe value to sound isn't just a (linear) bijection from one interval to another: it's a possibly messy map from one product of intervals (the probe value) to another product of intervals (the set of sliders in audpanel).

If you're lucky, you can just map the data-vector to the sound-parameters one scalar at a time. Most people haven't been trained to listen to sound one parameter at a time, though. Most generally, each sound-parameter should be some function of all the elements of the data-vector. You can find this by trial and error: starting with only one element of the data-vector, and introducing the others gradually, is one way. You can even leave out some of the elements -- nobody says you have to hear everything. The file exampleVectorProbe.c does this. (Tensor data are best reduced to vector data before sonifying.)

The nicest way to define such a multidimensional mapping in VSS is to use the HidimMapper Actor.

If you don't know when starting to run your application what minimum and maximum values a scalar might assume, the file exampleScalarAutorangingProbe.c implements a naive autoranging algorithm whose range is nonshrinking.


Noisy data may cause the sound to vary so much that patterns are obscured. The sound may communicate more if you somehow smooth the data before sonifying.

If the dataset is nominal instead of quantitative, for each "name" play one of a discrete family of sounds (probably not from a linear continuum; maybe individual sample files.) Sounds for ordinal data can obviously be taken from a linear continuum.

If the dataset is a sparse set of points, not a regular grid or lattice, you can make a sound corresponding to the distance from the probe to the nearest point. (A quiet Geiger-counter clicking works well for this, and needs no explanation to the user.) If the data is unevenly distributed in space, you can make a sound corresponding to the density of data points near the probe's position.

Searching can have differing scope. Early on you may be looking for large features, then narrowing in on smaller ones. So far, we've implicitly assumed that the data value to be sonified comes from a single point at the probe. But it can also come from integrating (averaging) all the data in a sphere centered on the probe's position, of varying radius. Points near the boundary of the sphere should contribute less than points near the probe itself. If the data is fairly smooth, a simple method is to average the values of several points randomly chosen near the probe.
If there is no explicit description of scope in the user interface, you can usually guess how wide the user's attention is by how large the probe's motion is. Fancy technique: smoothly vary the sphere's radius based on how much the probe has moved in the last (say) 10 seconds. The sound then automatically adjusts as the user switches between scanning the data, identifying features, and probing the data in detail.

Another way to sonify a region of data instead of a single data point is to sample points throughout the region and sonify them individually. The individual sounds should be very short, probably between 30 and 100 milliseconds, lest so many overlap and play at once that hard clipping ensues. The Granulation Actor is relevant to this problem.

Once you've finally found and selected the data you want, you typically perform some computation on the data to analyze or modify it. If the computation task itself takes some time, you can make the time pass faster for the user with a sonic progress indicator: a sound aiming in at a predictable target.

[ back ] [ up to main page ]