From The Economist - April 27 - 2019
Science & technology
A real brain wave
How to give voice to the speechless
Of the many memorable things about Stephen Hawking, perhaps the most memorable of all was his conversation. The amyotrophic lateral sclerosis that confined him to a wheelchair also stopped him talking, so instead a computer synthesised what became a world-famous voice.
It was, though, a laborious process. Hawking had to twitch a muscle in his cheek to control a computer that helped him build up sentences, word by word. Others who have lost the ability to speak because of disease, Or a stroke, can similarly use head or eye movements to control computer cursors to select letters and spell out words. But, at their best, users of these methods struggle to produce more than ten words a minute. That is far slower than the average rate of natural speech, around 150 words a minute.
A better way to communicate would be to read the brain of a paralysed person directly and then translate those readings into synthetic speech. And a study published in Nature this week, by Edward Chang, a neurosurgeon at the University of California, San Francisco, describes just such a technique. Speaking requires the precise control of almost 100 muscles in the lips, jaW, tongue and throat to produce the characteristic breaths and sounds that make up sentences. By measuring the brain signals that control these vocal-tract muscles, Dr Chang has been able to use a computer to synthesise speech accurately.
The volunteers for Dr Chang's study were Ave people with epilepsy who had had electrodes implanted into their brains as part of their treatment. He and his colleagues used these electrodes to record the volunteers' brain activity whitf those volunteers spoke several hundrejj sentences out loud. Specifically, the researchers tracked activity in parts of the brain responsible for controlling the muscles of the vocal tract.
To convert those signals into speech they did two things. First, they trained a computer program to recognise what the signals meant. They did this by feeding the program simultaneously with output from the electrodes and with representations of the shapes the vocal tract adopts when speaking the test sentences—data known from decades of study of voices. Then, when the program had learned the relevant associations, they used it to translate electrode signals into vocal-tract configurations, and thus into sound.
The principle proved, Dr Chang and his team went on to show that their system could synthesise speech even when a volunteer mimed sentences, rather than speaking them out loud. Although the accuracy was not as good, this is an important further step. A practical device that might serve the needs of people like Hawking would need to respond to brain signals which moved few or no muscles at all. Miming is a stepping stone to that. The team have also shown that the relationship between brain signals and speech is sufficiently similar from person to person for their approach to be employed to create a generic template that a user could finetune. That, too, will ease the process of making the technique practical.
So far, Dr Chang has worked with people able to speak normally. The next stage will be to ask whether his system can work for those who cannot speak. There is reason for cautious optimism here. What Dr Chang is doing is analogous to the now well-established field of using brain-computer interfaces to allow paralysed individuals to control limb movements simply by thinking about what it is they want to do. Restoring speech is a more complex task than moving limbs—but sufficiently similar in principle tp give hope to those now in a position similar to that once endured by the late Dr Hawking. ■
WOW…… WHAT THE MIND OF MAN CAN INVENT, TRULY ASTOUNDING - Keith Hunt