How the brain can control prosthetic hand movements

This article is published in collaboration with Caltech.

Neural prosthetic devices, which include small electrode arrays implanted in the brain, can allow paralyzed patients to control the movement of a robotic limb, whether that limb is attached to the individual or not. In May 2015, researchers at Caltech, USC, and Rancho Los Amigos National Rehabilitation Center reported the first successful clinical trial of such an implant in a part of the brain that translates intention—the goal to be accomplished through a movement (for example, “I want to reach to the water bottle for a drink”)—into the smooth and fluid motions of a robotic limb. Now, the researchers, led by Richard Andersen, the James G. Boswell Professor of Neuroscience, report that individual neurons in that brain region, known as the posterior parietal cortex (PPC), encode entire hand shapes which can be used for grasping—as when shaking someone’s hand—and hand shapes not directly related to grasping, such as the gestures people make when speaking.

Most neuroprostheses are implanted in the motor cortex, the part of the brain controlling limb motion. But the movement of these robotic arms are jerky, probably due to the complicated mechanics for controlling muscle movement. Having eliminated that problem by implanting the device in the PPC, the brain region that encodes the intent, led Andersen and colleagues to further investigate the role specific neurons play in this part of the brain.

The research appears in the November 18 issue of the Journal of Neuroscience.

“The human hand has the ability to do numerous complex operations beyond just grasping,” says Christian Klaes, a postdoctoral fellow at Caltech and first author of the paper. “We gesture when we speak, we manipulate objects, we use sign language to communicate with the hearing impaired. Tetraplegic patients rate hand and arm function to be of the highest importance to have better control over their environment. So our ultimate goal is to improve the range of neuroprostheses using control signals from the PPC.

“The more precisely we can identify individual neurons involved with hand movements, the better the capability these robotic devices will provide. Ultimately, we hope to mimic in a robotic hand the same freedom of movement of the human hand.”

In the study, the researchers used the rock-paper-scissors game and a variation, rock-paper-scissors-lizard-Spock. The game, says Andersen, is “perfect” for this kind of research. “The addition of a lizard, depicted as a cartoon image of a lizard, and Spock—a picture of Leonard Nimoy in character—was to increase the repertoire of possible hand shapes available to our tetraplegic participant, Erik G. Sorto, whose limbs are completely paralyzed. We assigned a pinch gesture for the lizard and a spherical shape for Mr. Spock.”

The game was played in two phases, first rock-paper-scissors and then the expanded game with the lizard and Spock. In the task, Sorto was briefly shown an object on a screen that corresponded to one of the hand shapes—for example, a picture of a rock or Mr. Spock. The image was followed by a blank screen, and then text appeared instructing Sorto to imagine making the corresponding hand shape with his right hand—a fist for the rock, an open hand for paper, a scissors gesture for scissors, a pinch for the lizard, and a spherical shape (loosely analogous to the Vulcan salute) for Spock—and to say which visual image he had seen, as the neuroprosthetic device recorded the activity of neurons in the PPC.

The researchers were able to identify single neurons in the PPC that fired when Sorto was presented with an image of an object to be grasped—a rock, say—and identified a nearly completely separate class of neurons that responded when Sorto engaged in motor imagery (the mental planning and imagined execution of a movement without the subject actually trying to move the limb).

“We found two mostly separate populations of neurons in the PPC that show either visual responses or motor-imagery responses during the task, the former when Erik identified a cue and the latter when he imagined performing a corresponding hand shape,” says Andersen.

The researchers discovered that individual neurons in the PPC also responded to hand shapes that did not directly correspond to a grasp-related visual stimulus. The paper shape can be related to the initial opening of the hand to grasp a paper, and the rock closing the hand to grasp a rock—and in fact, these imagined hand shapes were used by Sorto to imagine opening a robotic hand by imagining paper and closing the robotic hand around an object by imagining rock. However, scissors, lizard, and Spock call for imagining hand gestures that are more abstract and iconic than those needed to grasp the visual objects, and suggests, says Andersen, that this area of the brain may also be involved in more general hand gestures, such as ones we use when talking, or for sign language.

The results of the trial were published in a paper titled, “Hand Shape Representations in the Human Posterior Parietal Cortex.” In addition to Andersen and Klaes, other authors on the study are Spencer Kellis, Tyson Aflalo, and Kelsie Pejsa from Caltech; Brian Lee, Christi Heck, and Charles Liu from USC; and Kathleen Shanfield, Stephanie Hayes-Jackson, and Mindy Aisen from Rancho Los Amigos National Rehabilitation Center.

Publication does not imply endorsement of views by the World Economic Forum.

To keep up with the Agenda subscribe to our weekly newsletter.

Author: Deborah Williams-Hedges is Senior Media Relations Manager at Caltech. 

Image: A man shakes hands with a robotic prosthetic hand in the Intel booth at the International Consumer Electronics show (CES) in Las Vegas, Nevada. REUTERS/Rick Wilking.