New brain technology allows monkeys to transcribe New York Times articles – and even Hamlet

New technology allowed monkeys to transcribe passages from the New York Times and Hamlet at a rate of 12 words per minute.

The method, developed by scientists Krishna Shenoy, a professor of electrical engineering at Stanford University, and postdoctoral fellow Paul Nuyujukian, directly reads brain signals to drive a cursor moving over a keyboard.

Earlier versions of the technology have already been tested successfully in people with paralysis, but the typing was slow and imprecise. This latest work tests improvements to the speed and accuracy of the technology that interprets brain signals and drives the cursor.

“Our results demonstrate that this interface may have great promise for use in people,” says Nuyujukian, who will join Stanford faculty as an assistant professor of bioengineering in 2017. “It enables a typing rate sufficient for a meaningful conversation.”

Implants vs. eye-tracking

Other approaches for helping people with movement disorders type involve tracking eye movements or, as in the case of Stephen Hawking, tracking movements of individual muscles in the face. However, these have limitations, and can require a degree of muscle control that might be difficult for some people. For example, Stephen Hawking wasn’t able to use eye-tracking software due to drooping eyelids and other people find eye-tracking technology tiring.

Directly reading brain signals could overcome some of these challenges and provide a way for people to communicate their thoughts and emotions.

This new technology involves a multi-electrode array implanted in the brain to directly read signals from a region that ordinarily directs hand and arm movements used to move a computer mouse.

It’s the algorithms for translating those signals and making letter selections that the team members have been improving. They had tested individual components of the updated technology in prior monkey studies but had never demonstrated the combined improvements in speed and accuracy.

“The interface we tested is exactly what a human would use,” Nuyujukian says. “What we had never quantified before was the typing rate that could be achieved.” Using these high-performing algorithms developed by Nuyujukian and his colleagues, the animals could type more than three times faster than with earlier approaches.

Typing with auto-complete

The monkeys testing the technology had been trained to type letters corresponding to what they see on a screen. For this study, the animals transcribed passages of New York Times articles or, in one example, Hamlet. The results, which appear in the Proceedings of IEEE, show that the technology allows a monkey to type with only its thoughts at a rate of up to 12 words per minute.

People using this system would likely type more slowly, the researchers say, while they think about what they want to communicate or how to spell words. People might also be in more distracting environments and in some cases could have additional impairments that slow the ultimate communication rate.

“What we cannot quantify is the cognitive load of figuring out what words you are trying to say,” Nuyujukian says.

Despite that, Nuyujukian says even a rate lower than the 12 words per minute achieved by monkeys would be a significant advance for people who aren’t otherwise able to communicate effectively or reliably.

“Also understand that we’re not using auto completion here like your smartphone does where it guesses your words for you,” Nuyujukian says. Eventually the technology could be paired with the kinds of world completion technology used by smartphones or tablets to improve typing speeds.

In addition to proving the technology, this study showed that the implanted sensor could be stable for several years. The animals had the implants used to test this and previous iterations of the technology for up to four years prior to this experiment, with no loss of performance or side effects in the animals.

Shenoy and Nuyujukian are part of the Brain-Machine Interface initiative of the Stanford Neurosciences Institute, which is working to develop this and other methods of interfacing technology directly with the brain. The team is running a clinical trial now, in conjunction with Jaimie Henderson, professor of neurosurgery, to test this latest interface in people.

If the group is successful, technologies for directly interpreting brain signals could create a new way for people with paralysis to move and communicate with loved ones.

Additional authors include Jonathan Kao, a graduate student in electrical engineering, and Stephen Ryu, a consulting professor in electrical engineering. Krishna Shenoy is also professor (by courtesy) of neurobiology and bioengineering and a member of Stanford Bio-X and the Stanford Neurosciences Institute.

Support for the work comes from the Stanford Medical Scholars Program, the Howard Hughes Medical Institute, Paul and Daisy Soros Fellowship, the National Science Foundation, the Christopher and Dana Reeve Paralysis Foundation, the Burroughs Welcome Fund, the Defense Advanced Research Projects Agency, and the US National Institutes of Health.