Big Talk
Researchers from Columbia University have created a system that translates thoughts into recognizable speech — and its potential impact on people who've lost their ability to speak is extraordinary.
"Our voices help connect us to our friends, family, and the world around us, which is why losing the power of one’s voice due to injury or disease is so devastating," researcher Nima Mesgarani said in a blog post. "With today’s study, we have a potential way to restore that power. We’ve shown that, with the right technology, these people’s thoughts could be decoded and understood by any listener."
Training Day
According to a paper published on Tuesday in the journal Scientific Reports, the Columbia team's research started with a computer algorithm capable of producing what sounds like human speech.
"This is the same technology used by Amazon Echo and Apple Siri to give verbal responses to our questions," Mesgarani explained.
Next, the researchers asked five epilepsy patients undergoing brain surgery to listen to someone speaking sentences while the researchers looked for neural patterns in the patients' brain activity. The Columbia team then used those patterns to train their algorithm, which they called a "vocoder," to translate brain waves into speech.
Listen Up
After that, it was time to test how well the device translates thoughts.
To do that, the researchers asked the epilepsy patients to listen to someone recite the numbers zero though nine while the team recorded their brain signals. The researchers ran the recordings through the algorithm, then had a specially trained neural network analyze and "clean up" the output.
Finally, the team asked 11 people to listen to the speech to test its intelligibility.
"We found that people could understand and repeat the sounds about 75% of the time, which is well above and beyond any previous attempts," Mesgarani said. "The sensitive vocoder and powerful neural networks represented the sounds the patients had originally listened to with surprising accuracy."
Mind Reader
Right now, the system only translates thoughts as a person listens to speech, but the researchers plan to see if they can replicate the study while having a person speak or think about speaking — no listening involved.
"In this scenario, if the wearer thinks ‘I need a glass of water,’ our system could take the brain signals generated by that thought, and turn them into synthesized, verbal speech," Mesgarani said. "This would be a game changer. It would give anyone who has lost their ability to speak, whether through injury or disease, the renewed chance to connect to the world around them."
READ MORE: Columbia Engineers Translate Brain Signals Directly Into Speech [Columbia University]
More on speech: New Brain Implant Could Translate Paralyzed People’s Thoughts Into Speech
Share This Article