AI Music is "colorless." Here's how we can fix it. | Benjamin Honeray Lee | TEDxTsinghua SIGS

Key Concepts

• Colorless Definition of Music: The reductionist view that music is merely sound following theoretical rules (rhythm, pitch, genre), ignoring the "color" of human context.
• Associations: The external factors (history, composer identity, intent) that imbue sound with emotional meaning.
• AI Performer Bias: The psychological phenomenon where listeners rate identical music differently based on whether they believe it was created by a human or an AI.
• Human-Centric Music Framework: A proposed model for AI music that incorporates three "primary colors": Human Beings, Human Effort, and Alleviation of Human Pain.

---

1. The "Grayscale" Problem in AI Music

The speaker, a computer scientist and musician, argues that current AI music research suffers from a "colorless" definition. By modeling music solely through technical parameters—such as genre adherence, vocal fidelity, and prompt alignment—researchers treat music like a grayscale image.

• The Analogy: Just as an AI trained only on grayscale data fails to represent the "color" of the real world, AI music models that ignore context fail to capture the essence of what makes music meaningful to humans.
• Evidence: Peer-reviewed studies demonstrate that listeners’ emotional responses change based on the perceived origin of the music, proving that the "story" behind the music is as important as the sound itself.

2. The Power of Associations

The speaker posits that music is defined by the associations we attach to it.

• Case Study: Erquan Yingyue (Moon's Reflection on the Second Spring) by Abing. The piece is deeply associated with the suffering of the Chinese people during the 1931–1945 war. The speaker notes that if the composer were revealed to be a Japanese soldier from that era, the emotional impact on the listener would shift from empathy to repulsion, despite the notes remaining identical.
• Psychological Impact: Research indicates that even changing lyrics to be more violent—without altering melody or rhythm—can increase aggression in listeners, proving that associations are not just abstract concepts but physiological triggers.

3. The Three Primary Colors of Music

To elevate AI music from a "colorless shell" to a true art form, the speaker proposes integrating three essential associations:

I. Associations to Human Beings (User Honesty)

• The Issue: Listeners view art as a mirror of unique human experience. When AI is used to deceive (pretending to be a human artist), it creates a disconnect.
• Actionable Insight: Transparency is required. AI music must be honest about its origins to maintain the listener's trust and connection to the "pulse" of the creator.

II. Associations to Human Effort (User Control)

• The Issue: Live performances and traditional music production are valued because of the human labor involved. Current AI models often bypass this by generating "finished" products.
• Actionable Insight: AI tools must evolve to allow for granular human control—modifying pitch, vibrato, dynamics, and orchestration—similar to how MIDI or Vocaloid software allows composers to inject their own effort into the creative process.

III. Associations to Alleviating Human Pain (Consent and Ethics)

• The Issue: The current practice of training AI on human works without credit or compensation creates a logical contradiction: using exploitative methods to create art intended to provide catharsis.
• Actionable Insight: The speaker draws a parallel to medicine (e.g., organ transplants), where life-saving procedures require informed consent. AI music must develop algorithms that trace, credit, and obtain consent from original artists to align with human well-being.

4. Synthesis and Conclusion

The speaker concludes that music is not merely a mathematical arrangement of sound, but a complex web of human history, effort, and ethics.

Notable Quote: "Double-blinded tests in AI music that prevent us from knowing about the origin of a song, the story of its composer, or even just general historical events, only forces AI music towards becoming a colorless shell of what music fully is."

Final Takeaway: For AI music to be considered "music" in the fullest sense, it must move beyond technical fidelity. It requires a framework of honesty, granular human control, and ethical consent. By reintroducing these "primary colors," we can ensure that technology serves to preserve, rather than replace, the human experience.