These Full-Body Deepfakes are Like Nothing We've Ever Seen

Face Off

Full-body deepfakes are here.

Deepfakes are manipulated and misleading videos that use deep learning technology to produce sophisticated doctored videos. Normally this involved splicing someone's face onto an existing video, often making it seem like a celebrity or public figure did something that never happened.

Now the tools exist to do the same thing with someone's entire body, like a digital puppeteer. That's thanks to a new technique out of Heidelberg University that was recently published to Github and presented at September's European Conference on Computer Vision — a step forward for deepfakes that has escaped mainstream scrutiny, but that could lead to the next generation of altered footage.

The technology isn't perfect yet, but it's a worrisome preview of what could come. It certainly impressed Károly Zsolnai-Fehér, the narrator of the YouTube channel Two Minute Papers, who enthusiastically described the work in a video about the research.

"This way, we can appear to be playing tennis, baseball, or finally be able to perform 100 chin-ups," he joked. "Well, at least on Instagram."

 

Body Double

Let's say you want to create a fake video of yourself crushing a game of tennis. With past iterations of deepfake technology, you'd be limited to the exact movements that occur in an existing video of a tennis pro. The resulting deepfake would show the tennis player's body going through the motions, but their face would be replaced with yours. In that sense, the tennis pro would be "driving" the video with your face edited in for the ride.

But this new algorithm isn't bound by that constraint. It can learn what you look like by examining existing imagery of you, then map you — outfit and all — onto footage of the tennis pro. It could even place you into a simulated virtual environment and make you amble around, moving your arms and legs like a marionette.

Even more impressively, the new system doesn't need to see its subject from every angle to model their appearance. Let's say you want a video of yourself turning around to land a wicked backhand swing, but you never faced away from the camera in the imagery you fed to the AI system. This new system can guess what you look like from that angle based on the system's own contextual understanding of what a human body looks like — down to details, Zsolnai-Fehér points out, like a belt extending around a person's waist.

Stop and Think

The researchers behind this project are excited about how the tech could revolutionize video games or animated movies — though it could also be used for political manipulation or other nefarious purposes.

"Generally speaking, any improvement by the research community on the topic of image synthesis carries the inherent risk of being misused," Björn Ommer, a computer vision researcher at Heidelberg who worked on the new algorithm, told Futurism.

Fortunately, this new algorithm's simulations are still choppy — a subject's legs will thicken and narrow as they walk in a circle, and an inability to correct shadows or differences in lighting between sample data and the generated simulations create glitch-in-the-matrix style auras around the deepfake's subject.

Ommer suggests that these flaws could serve as indicators for deepfake-detecting algorithms.

"Discriminating fakes from real images is an easier task than generating these images. After all, the competition between generation and discrimination is how the images are synthesized," Ommer told Futurism. "Thus, it seems likely that models for detecting synthesized fakes will be able to keep up."

But that presumes that any deepfake-detecting algorithm is trained on the same techniques used to generate the deceptive video in the first place. And now that this technology is out there, it's safe to assume someone else is out there working out all these bugs.

READ MORE: This AI Learns Human Movement From Videos [YouTube]

More on Deepfakes: If DARPA Wants To Stop Deepfakes, They Should Talk To Facebook And Google