When the COVID-19 Pandemic struck, musicians still strove to perform together and spread hope and joy. The experience of performing virtually rather than physically quickly proved difficult through video conferencing applications, such as Zoom, with video delays and other limitations.
Three U-M associate professors are hoping to change that. Michael Gurevich, associate professor in the Department of Performing Arts Technology (PAT), John Granzow, associate professor in the PAT department, and Matt Albert, associate professor and chair of the Department of Chamber Music, have been developing their Visualizing Telematic Performance project which explores ways to incorporate visual elements and imitate the presence of multiple musicians – for the benefit of both performers and audience – when players are separated geographically.
Beginning in January 2021, the researchers on this project began to examine ways to create three-dimensional objects– called robots, mechanical avatars, or kinetic sculptures –that would be on a stage with performers, representing the movements of other performers in a different location.
The movements of the live musician are recorded through a suit with tracking dots on the joints and limbs. The robots are then able to reflect the movements of a musician’s body in real time onstage, thanks to a set of code that translates the performer’s motions into instructions for the robot to match.
The team was able to see their efforts become a reality in December of 2022 at a telematic concert held simultaneously between the University of Michigan in Ann Arbor and the University of Virginia (UVA) in Charlottesville. Each school hosted their own performer, wearing a motion capture suit, and a robot representative of the fellow school. Both audiences were able to experience a performance of live duets from the two musicians.
While the team’s research aims to help study how performers and audiences relate to the robots, Gurevich stated that, “the deeper goal is really to help develop perspective and understanding on how performers move.”
Student Adam Schmidt agreed, “it’s been so interesting to investigate and try to understand the way musicians communicate non-verbally with their bodies.”
Schmidt graduated in 2022 with degrees in sound and electrical engineering, and is currently pursuing an MA in media arts while working with the Visualizing Telematic Performance team. “We are hoping to deepen our understanding of what it means to communicate musically from musician to musician as well as from musician to audience.”
Article summarized from U-M SMTD’s Michigan Muse: Expanding Horizons: A Look Inside the Performing Arts Technology Department – University of Michigan School of Music, Theatre & Dance
