Examples of audio traces captured using a microphone close to an LCD display with varying “zebra stripe” patterns on-screen. (Genkin, Pattani, Schuster, Tromer)
Ever wonder what the people on the other end of a Hangouts session are really looking at on their screens? With a little help from machine learning, you might be able to take a peek over their shoulders. Based on research published by Daniel Genkin, assistant professor of electrical engineering and computer science at the College of Engineering, all you’ll need to do is process the audio picked up by their microphones. Genkin is part of an international team of researchers that investigated a potential new avenue of remote surveillance that they have dubbed “Synesthesia”: a side-channel attack that can reveal the contents of a remote screen, providing access to potentially sensitive information, based solely on “content-dependent acoustic leakage from LCD screens.” Their research was published in August at the CRYPTO 2018 conference in Santa Barbara, CA.
