USC engineering class creates tools to analyze musical expression

May 11, 2006

Student Eric Cheng's analysis of dynamic shaping of performances of Bach's Sonata #2 for unaccompanied violin by Yehudi Menuhin (blue) and Nathan Milstein (green). Credit: Eric Cheng

Music expresses and elicits emotion. But how, exactly? Philosophers have been fascinated by the question since Pythagoras. At the USC Viterbi School of Engineering, Elaine Chew, assistant professor of industrial and systems engineering, now teaches a graduate course on using computational and other engineering tools to look for answers.

Chew, who continues a career as a distinguished concert pianist in addition to her engineering research, has written an account of the issues in creating the course in an article that will be presented at an engineering conference later this year.

Meanwhile the students in "Computational Modeling of Expressive Performance" presented their results May 3, a collection of 10 projects now up on the class website as what Chew calls a "non-peer reviewed publication" at http://www-scf.usc.edu/~ise575/b/projects

The projects from the class, offered for only the fourth time this Spring, range widely. Arpi Mardirossian found a trove of silent movie scores specially written to evoke specific emotions, and analyzed their characteristics.

For "Analysis of Dynamic Shaping in Unaccompanied Bach," student Eric Cheng created intricate graphs comparing, note-by-note, performances of a Bach unaccompanied violin sonata by three master violinists: Jascha Heifetz, Yehudi Menuhin, and Nathan Milstein, with respect to their dynamic shaping, i.e. the nature and amount of sound intensity variation.

Meghen Miles and Merrick Mosst created "Emotiongrams" by mapping specific musical characteristics (e.g., minor keys), widely identified with certain emotions (e.g., sadness), to color patterns that represented varying combinations of energy and stress.

The other seven topics pose equally interdisciplinary questions - questions that are quite new in engineering in general, and engineering education in particular.

In a recently accepted conference publication that will be presented in fall in San Diego, "A case study in course design at the intersection of music and engineering," Chew details the challenges in creating the class, and discusses the problems that remain.

Chew traces the beginning of modern efforts to bring engineering techniques into analysis of music to the work of Christopher Longuet-Huggins, a noted theoretical chemist and cognitive scientist, and gifted amateur musician, who in his "Letters to a Musical Friend" (The Musical Review, 1962) described computing methods for "Interpreting Bach" that were implemented and published in Machine Intelligence in 1971.

While interest in the field has mushroomed in the 21st century, with engineering conferences and refereed journals now covering the subject, teaching is only beginning. "The challenges include the lack of a formal body of knowledge, in the form of a text, the lack for formal academic structures to support the course, the lack of students with suitably strong backgrounds in both computing and music, and misconceptions about the nature of music research," she notes in her presentation.

Attacking problems in computational modeling of music "draws upon methodologies and tools from music theory, cognitive science, artificial intelligence, experimental psychology, mathematics, signal processing and neuroscience. Few if any students enrolled in the course are equipped with the knowledge to understand all the material."

The syllabus devised offers a crash course in the elements of all these disciplines touching on music cognition, with the aim that, at the end of the course, each student should be able to "understand basic music structures; be capable of manipulating digital music; be able to generate computational means of analyzing, generating and visualizing structured music; and be able to formulate a question and build the computational tools to answer it."

The course's final projects depend on the students having acquired at least the rudiments of all these skills. The amount of material that has to be mastered is so great that Chew reluctantly had to move what had been a popular feature of the course - guest appearances by musicians and researchers in the field - to its own separate series.

Precisely because of the unusual mixture of disciplines involved, making this course a reality was not institutionally easy, Chew says. However, a remarkable record of recognition for her research (she holds a Ph.D. from M.I.T in engineering, and a FTCL from Trinity College, London in piano performance, an Early Career / PECASE award from the National Science Foundation, and the Viterbi Early Career Chair from USC) helped to pave the way.

"All these interdisciplinary education and outreach activities could not have happened without the staunch support of my own department, the deans of the Viterbi School, and the provost," Chew says. "My own department has graciously allowed me to create this special topics class in lieu of teaching another traditional industrial and systems engineering course. My department chair, James Moore, has encouraged me to forge ahead in creating an undergraduate counterpart to the class because he sees the potential of such courses in recruitment and retention of young and inquiring minds to engineering."

Says Moore, chair of the Daniel Epstein Department of Industrial and Systems Engineering, her professional home: "Taking interdisciplinary work seriously involves accepting some intellectual risks. Her work is novel, but important; and this makes her course novel and important. It is a source of no small pride for us that the most appropriate avenue for connecting Prof. Chew, her students, and her work to the USC Viterbi School is the Daniel J. Epstein ISE Department."

With this kind of class, it's not just a matter of "if you build it, they will come. Recruiting students was a major effort.

"I personally and actively publicized the class widely through emails and posters to other departments in the Viterbi (Electrical Engineering, Computer Science, Bioengineering) and Thornton schools (Composition, History and Literature, Education), and the College (Psychology, Neuroscience, Mathematics)."

What kind of student winds up in a class like this? "This year, the class consisted of students from ISE, EE, CS, and Neuroscience; we had a vocalist, a violinist, a pianist, some with a couple years of instrumental training, and a few with no musical background."

For example, Meghen Miles: "I have a Bachelors' degree from the Thornton School of Music, and now I'm getting a Ph.D. in Neuroscience. I found out about the class from emails from quite a few sources, including Elaine Chew herself. I thought the class would help me both with my understanding of computational modeling and by introducing me to a new body of literature with direct ties to my research.

"The structuring of the class really forced me to actually read the articles, even when I didn't feel like they applied to my own research, I now have an enriched understanding of current and past research in the expressive performance field. Although it's not all directly applicable to my own work, it's given me some very good ideas."

So: how does music express and evoke emotion? Elaine Chew and her students don't know yet. But check back in a few years.

Source: University of Southern California