This Robot Conducting an Orchestra Mimics Human Movements but Lacks Human Soul

YuMi conducting in rehearsal under the watchful eye of Andrea Colombini. YouTube

Verdi and Puccini never saw this coming.

Last night the Lucca Philharmonic Orchestra performed in Pisa, Italy for a crowd of about 800 people. Grammy-nominated tenor Andrea Bocelli and soprano Maria Luigia Borsi also sang at the gala.

It was a perfectly normal symphony performance aside from one crucial aspect: it was conducted by a robot.

YuMi (short for “you and me working together”) was manufactured by Swiss robotics company ABB. It was originally designed as an assembly bot—its earliest tasks involved manufacturing watches, phones and tablets.

The 84-pound robot is equipped with 30 sensors that perceive the world in 3D. It also has arms, elbows and wrists which allow for freedom of motion. Its fluid movements are similar to those of a human, which is why ABB decided to give YuMi a musical test during this week’s First International Festival of Robotics in Pisa.

Conductor Andrea Colombini trained YuMi using a method called lead-through programming—he literally moved the robot’s arms up and down in tune with the music so the detailed motions could be recorded in YuMi’s computer memory. Here’s what that training looked like in action:

ABB then fine-tuned YuMi’s movements, using its own software to synchronize the motions and music.

“The gestural nuances of a conductor have been fully reproduced at a level that was previously unthinkable to me,” Colombini wrote in a blog post.

YuMi conducted three pieces during last night’s concert: “La Donna è Mobile” from Verdi’s Rigoletto, “O mio babbino caro” from Puccini’s Gianni Schicchi and the “Intermezzo” from Mascagni’s Cavalleria Rusticana.

Itzhak Perlman shouldn’t worry just yet, however: Colombini pointed out that YuMi wasn’t programmed to respond to changes in an orchestra’s performance. For example, if a certain player fell behind the beat or missed a note, YuMi would have no way of correcting them.

“It is ultimately not gifted with human sensitivity,” Colombini wrote. “The robot uses its arms, but the soul, the spirit, always comes from a human.”

According to Colombini, YuMi should supplement or assist human conductors rather than replace them.

“The director makes the adjustments that result in the material and artistic interpretation of a work of music,” he concluded.