AI-Generated Sound Effects Are Now Fooling Human Ears

A small study found people were tricked by sophisticated audio AI.

Turing Test

If you'll permit us to spoil a little bit of movie magic, many of the sound effects you hear in film and TV are actually recreated and edited in later by Foley artists. Now, researchers are attempting to create sound effect-generating artificial intelligence to see if they can do their jobs well enough to fool the general population.

In a recent study, a small cohort of participants fell for the trick: Most they believed that the AI-generated noises were real, IEEE Spectrum reports. Sometimes, they even chose the AI version over a video's original audio.

 

Sleight Of Hand

In the study, which was published in June in the paper IEEE Transactions on Multimedia, 41 of the 53 participants were fooled by the AI-generated sounds.

That sounds damning for the Foley artists who recreate sound effects in the studio, but for now, the algorithm's still far from perfect.

 

Sleight of Ear

Where the AI currently falters — and the challenge going forward — is figuring out how to match the audio up to the actual video it supposedly represents. In order to fool an audience, the sound effects need to not only sound real, but also, synch up with a video's action.

That's why the embedded videos ­— spoilers, they're all AI-generated — of rainfall or a crackling fire are convincing but this one of a running horse, where the clipping and clopping clearly don't line up with the steps the horse takes, is a clear fake. Needless to say: The jobs of foley artists and sound editors are secure — for now. But it's hard not to imagine a not-at-all distant reality where you might believe this horse is making these horse noises:

 

READ MORE: New AI Dupes Humans into Believing Synthesized Sound Effects Are Real [IEEE Spectrum]

More on AI: Elon Musk: If You Don't think AI Could Outsmart You, You're an Idiot