How can microbes be sonified? Once inoculated, our microbiome records were imaged utilizing an open-source camera to generate 2D images. We developed several approaches to generate sound and music from these images, with a principal question in mind: how is the sound meaningful from a biological perspective?

Ultimately, we focused on three forms of data that could be extracted from the images of our records: microbe location, density, and cluster size. Given the design of our records, we could add an additional layer of information by having different segments of each record correspond to organisms swabbed from different parts of the body. With our incubator and camera, we could also take time-lapse images, adding a time parameter. Based on these inputs, we developed several algorithms to convert these data streams to sound and music.

Sonification: the use of non-speech audio to convey information or perceptualize data.

Auditory perception has advantages in temporal, spatial, amplitude, and frequency resolution that open possibilities as an alternative complement to visualisation techniques.

There is prior work sonifying algae data, star data, and microbial data into music by various methods. For example, scientists looked for patterns in readings from blue-green algae samples collected in the western English Channel then transformed the data into musical notes: http://www.livescience.com/23626-microbe-music-algae-songs.html. The tune "Bloom" illustrates how some algae species bloom occasionally and become more abundant for short periods of time. They had lower abundance of microbe correspond to lower notes, and chord progression was related to day length, chlorophyll concentration in the water, and other physical parameters.

Here astronomers converted star data from NASA's Kepler telescope into reggae songs: link.

Then a team converted RNA sequences and protein 3D structure into sound: link.

Based on inspiration from the previous sonification projects, the sonification software was designed using prior image processing techniques. First, a Gaussian filter reduces the image noise, and the image is converted to a black-and-white binary image. The location of the center of each cluster and the size of each cluster are stored. Based on this information, the density of each area on the plate is computed.

Each image is mapped to a coordinate system, and the colony location correlates to musical notes, or pitches. Then density of the colonies corresponds to background music. Granular Synthesis is a method by which sounds are broken into tiny grains that are redistributed and reorganized to form other sounds. Here the concept of granular synthesis is used with the density data.

In the first attempt, shape and size of the colonies was to correlate to different scale types: major, natural minor, melodic minor, pentatonic,and chromatic. However, a classifier with only five groups brings more ambiguities. Thus, it was switched to FM synthesis. FM synthesis stands for frequency modulation synthesis; the timbre (tone quality) of a waveform is changed by adjustingits frequency with a modulator frequency that is also in the audio range, resulting in a more complex waveform. This new tone can be described as "gritty" if it is a thick and dark timbre. For synthesizing harmonic sounds, the modulating signal must have a harmonic relationship to the original carrier signal.

With two types of culture plates, the program can be used to compare the sonification results from each. With a timelapse video for 2-3 days of the culture plates, it is also possible to listen to the sound change as the microorganisms grow.

Setup for camera module: