You’re practicing a new song on piano. One part keeps tripping you up. No matter how many times you try it, your fingers just won’t move fast enough. Then you put on a special device that looks like a robotic hand. Its motor wiggles your fingers faster than you could move them on your own. When you sit at the piano again without the robo-hand, your fingers can now hop into high gear. You’re playing the fast part that seemed impossible before.
In a recent series of experiments, dozens of professional piano players had this exact experience. They improved their piano skills using a wearable robotic hand, also called an exoskeleton. Shinichi Furuya and his team shared the results January 15 in Science Robotics.
“Many people are using this kind of robot for rehabilitation or virtual reality gaming,” says Furuya. “I thought maybe this is also good for music education.” Furuya is a researcher at the NeuroPiano Institute at Sony Computer Science Laboratories, Inc. in Tokyo, Japan.
Nicholas Hatsopoulos studies sensory motor control at the University of Chicago in Illinois. A neuroscientist, he did not take part in the new study. But he really liked it. “I thought it was super cool,” he says.
A new way to practice
Furuya used to be a professional musician. “I practiced [piano], like 10 hours every day,” he says. He practiced so much that he developed a movement disorder in his hands. It’s why, he says, “I changed my career from pianist to scientist.”
His hands are doing better now. And he still plays and teaches piano. But a few years ago he also started imagining a new way for musicians to practice. “I was thinking maybe we need some kind of a robot that can teach how to move the fingers,” he says.
Furuya is not an engineer. So he told one of his colleagues about his idea. He wondered if it might be possible to build such a robot. And he was told, “Yeah, why not?”
The robot they built fits over one hand. To test the device, Furuya and his colleagues brought professional pianists into the lab. The number of subjects was “very impressive,” says Hatsopoulos.
For one experiment, 60 pianists practiced the same complex pattern. It involves pressing down the index and ring finger together, then the middle finger and pinky together. You switch back and forth as fast as possible. Try it!
Even moving very slowly, this is super tricky. The pianists practiced until their fingers couldn’t move any faster.
The researchers then divided the 60 pianists into five groups. The first group wore the robo-hand as they played that tricky pattern. It moved the fingers faster than the musicians could manage on their own. For the second group, the device moved the fingers slowly in the same tricky pattern. The third group experienced rapid finger motion, but all the fingers moved at once. Two other groups didn’t use the robo-hand. One of these continued to practice the tricky pattern on their own. The final group just rested.
Afterward, only one group could play the tricky pattern faster than before. These were the pianists who used the robotic hand to practice the complex finger motions quickly. The time between their keystrokes sped up by around 6 percent, on average.
The pianists often told Furuya that their fingers felt “very light” after using the robot. Also, they only used the device on their right hands. Yet their left hand’s speed also improved!
“I was stunned by that, actually,” says Hatsopoulos.
One day after training with the robot, these pianists could still play faster than usual.
Furuya doesn’t expect the effect to last much longer than that, though. He thinks pianists would probably have to use the device regularly to maintain the speed bonus.
Alas, they can’t do this now. “This is still a prototype,” explains Furuya, so they can’t yet take the device home with them.
Hatsopoulos thinks this robotic training likely rewired some neural connections in the musicians’ brains. The somatosensory (So-MAT-oh-SEN-or-ee) cortex is the brain region responsible for handling sensory feedback. It’s what gets activated as the robot wiggles people’s fingers.
When someone wiggles their own fingers, a different part of the brain — the motor cortex — manages that motion.
But these two parts of the brain share information, says Hatsopoulos. So do the left and right sides of the brain, which control the opposite sides of the body, he notes. These connections could help explain why the training improved the musicians’ speed in their left hands, even though they only wore the device on their right.
Therapy, sports, gaming and beyond
Doctors and researchers already use robotic devices similar to this one in therapy, notes Hatsopoulos. For example, a stroke often impairs movement on one side of the body. Using a robot to move an arm can help a patient get back their former range of motion.
Exoskeletons that move fingers and hands have lagged behind, though. That’s because robotic fingers are complex and “harder to build,” says Hatsopoulos.
The robot hand Furuya’s team built can only move fingers one way — in a downward motion. “We are developing a more complex robot,” he says. It will be able to move two joints at once. That will allow for even more complex finger movements.
Furuya imagines that robotic training could help people master many activities beyond piano keys. Pro gamers must practice complex keyboard or joystick motions. Surgeons have to learn complicated, delicate hand movements. And baseball pitchers need to master gripping the ball in a certain way. If a robot can help someone train these sorts of hand motions, Furuya says, maybe practice won’t have to feel so repetitive. “They can use practice time for more creative activities,” he says.