It could make it easy for anyone to create deepfakes.
On Thursday, a U.K.-based tech firm released a remarkable demo video of its flagship software.
"I'd like to demonstrate how Synthesia technology can make me speak 7,100 languages fluently," says the smiling person in the clip. A few seconds later, a digitally manipulated version of the clip shows the same person saying the same sentence in French, Mandarin, and Portuguese.
But while the clip seems lighthearted enough, the technology powering it could be anything but — in fact, some see it as a direct threat to democracy.
Your Words, Not Mine
According to Synthesia's website, the purpose of its AI-powered dubbing tech is to create a world without language barriers in which anyone can enjoy any video content, regardless of its language of origin.
"We hope that this new medium will foster cultural exchange, joy and deeper understanding in the same manner that the written word has done for centuries," the company writes on its homepage.
Others are far less idealistic. They envision a future in which malicious actors use software like Synthesia's to create deepfakes — videos in which a person appears to say something they didn't actually say.
“By blurring the line between fact and fiction, deep fake technology could undermine public trust in recorded images and videos as objective depictions of reality,” three members of Congress wrote in a letter to Daniel Coats, the U.S. director of national intelligence, in September.
The Deep End
Examples of deepfakes are already online, but they're typically either not that realistic-looking or created by trained digital artists. Synthesia's AI-powered software could give anyone with a computer the ability to create their own convincing deepfakes.
While Synthesia has acknowledged this possibility, the company doesn't appear too troubled by it. “We are trying to solve a very complex and technical problem,” chief executive Victor Riparbelli told the Telegraph. “We are not releasing any software to the public…There is a wider discussion to be had about the malevolent use of this kind of stuff.”
Of course, not actively releasing software to the public and actually keeping it out of the public's hands are two very different things — and if this particular software gets into the wrong hands, there's no telling what kind of devastation it could cause.
READ MORE: Tech That 'Threatens Democracy' Is Being Funded by UK Taxpayers [The Telegraph]