What if our data sounded like music? | Jonathan Middleton | TEDxSpokane

Key Concepts:

• Sonification: Representing data as sound.
• Auditory Display of Data: Using sound to communicate information derived from data.
• Algorithmic Composition: Using algorithms to translate data into musical parameters.
• Data-Driven Music: Music created based on data inputs.
• User Experience (UX) in Data Science: Designing data representations to be engaging and informative for the user.
• DNA Sonification: Representing DNA sequences as music.

1. Introduction and Meteorological Data Sonification

• The speaker, a classically trained composer, introduces the concept of composing music from data, specifically meteorological data.
• The initial example is a weather forecast sonified, aiming to convey whether it will be rainy, sunny, or cloudy.
• The speaker contrasts visual data displays (icons) with auditory displays, noting that icons are not temporal, unlike music and weather.
• The sonified weather forecast uses high, glistening sounds to represent sunshine and a rising line in the cello, derived from atmospheric pressure, to indicate more sunny days.

2. Algorithmic Composition and User Engagement

• The process of translating data into music involves a "balancing act" between pure data representation and increased musicality to capture deeper meaning.
• Algorithmic composition involves feeding data through an algorithm that outputs associated pitches, rhythms, loudness, or other musical traits.
• Auditory displays are effective because they invite and engage users, which is crucial for user experience in data science.
• Example: Composed weather forecasts as a morning alarm, indicating whether to grab a hat and coat (cold) or sunscreen (sunny).

3. Sonification of Physiological Data: Trail Runner Example

• An auditory display of a trail runner's heart rate is presented, with the marimba following the contours of the heart rate data, highlighting peaks and valleys.
• The speaker explores increasing musicality to capture the intensity of a trail run, incorporating heart rate, vertical speed, cadence, and altitude data.
• The data is summarized and looped to create a song-like feel, with the heart rate data represented in the marimba.

4. DNA Sonification: Redwood Symphony

• The concept of hearing the sounds of a redwood tree, including the wind through its branches and the creaking of wood, is introduced.
• The focus shifts to hearing the molecular structure of the tree, specifically its DNA.
• A fragment of the redwood tree's genetic code (nucleotides A, T, C, and G) is sonified.
• The raw data is presented with animation, followed by an orchestrated version called "Redwood Symphony," embedding fragments of the DNA sequence.
• The C's in the DNA sequence are mapped to the pitch zero, representing silence, where silence signifies something rather than nothing.

5. Quotes and Conclusion

• The speaker shares quotes from Dr. Robert Bywater regarding their work with protein structures.
• Quote 1 (from a New York Times interview): "It's like Allan Turing solving the Enigma codes. You get a message you do not understand it, you have to convert it into something you do understand."
• Quote 2 (from an email): "What the music will do is provide us a way to recognize, remember, and compare these complex objects [protein structures]. I think that will be extremely powerful."
• The speaker argues that using musical characteristics to represent data provides a new dimension and new experiences with data.
• Music has many roles in society, designed to motivate and engage us, and is ubiquitous like data.
• The conclusion is that we should consider hearing data through music.

Technical Terms and Concepts:

• Sonification: The process of transforming data into auditory representations.
• Algorithmic Composition: The technique of using algorithms to generate music based on data inputs.
• Nucleotides (A, T, C, G): The building blocks of DNA, represented musically in the Redwood Symphony.
• Pitch Zero: In this context, silence, used to represent the nucleotide "C" in the DNA sequence, signifying meaningful absence.

Logical Connections:

• The presentation moves from simple meteorological data to more complex physiological and genetic data.
• Each example builds upon the previous one, demonstrating the increasing sophistication and potential of data sonification.
• The quotes from Dr. Bywater provide a theoretical and philosophical underpinning for the practical examples.

Synthesis/Conclusion:

The presentation effectively demonstrates the power of sonification as a tool for understanding and engaging with data. By translating complex datasets into musical forms, the speaker shows how we can gain new insights and experiences. The examples, ranging from weather forecasts to DNA sequences, highlight the versatility of this approach. The key takeaway is that music, like data, is a ubiquitous and powerful medium that can be harnessed to enhance our understanding of the world around us.