The present work is the first that decodes continuously spoken speech and transforms it into a textual representation. For this purpose, cortical information is combined with linguistic knowledge and machine learning algorithms to extract the most likely word sequence.
The brain activity was recorded in the USA from 7 epileptic patients, who participated voluntarily in the study during their clinical treatments. An electrode array was placed on the surface of the cerebral cortex (electrocorticography (ECoG)) for their neurological treatment.
While patients read aloud sample texts, the ECoG signals were recorded with high resolution in time and space. Later on, the researchers in Karlsruhe analyzed the data to develop Brain-to-Text.