All spies, listen up: You no longer need to hide a mic. It appears that a team of MIT researchers have managed to turn a potato chip bag into a eavesdropping listening device.
In fact, their technique, which extracts audio by visually analyzing the tiny vibrations triggered when sound strikes the surface of an object, works on just about any object in a room, from a potted plant to the surface of a glass of water.
In a series of experiments, nicely detailed in this project video, the research team was able to reconstruct music and spoken words using a high-speed camera behind a soundproof window. By applying a complex algorithm to the visual information captured by the camera, the system can “listen” to sounds generated in the room. The vibrations being recorded are tiny indeed — on the potted plant, for instance, the vibrations moved the plant surface about a tenth of micrometer on average.
The team — including researchers from MIT, Microsoft and Adobe — will present their findings in a paper at this year’s Siggraph computer graphics conference.
“When sound hits an object, it causes the object to vibrate,” says graduate student and lead author Abe Davis, on the MIT project page. “The motion of this vibration creates a very subtle visual signal that’s usually invisible to the naked eye. People didn’t realize that this information was there.”
In order to reconstruct audio from video, the frame rate of the video sample must be higher than the frequency of the audio signal. As such, the researchers used high-speed cameras able to capture 2,000 to 6,000 frames per second.
But in the process of testing the technique, the team discovered they were also able to extract limited audio information from standard smartphone cameras, which record at 60 frames per second. While the high-speed camera system reproduces sound with surprising fidelity, the standard camera system can provide more limited information — the number of people speaking in a room, say, and possibly gender or other identifying information.
The “visual microphone” technique has obvious eavesdropping applications in forensics and law enforcement. But the researchers say they’re more interested in developing a new kind of imaging in which scientists can determine structural properties of objects from their visible response to bursts of sound.
“I’m sure there will be applications that nobody will expect,” says researcher Alexei Efros. “I think the hallmark of good science is when you do something just because it’s cool and then somebody turns around and uses it for something you never imagined.”