Ambient light sensor exposes screen privacy issue

Getting your Trinity Audio player ready...

If you’ve taped over your laptop camera, or have a more elegant privacy slider, then you’ll already be aware of how prying electronic eyes can spy on victims. Modern, high-resolution, front-facing cameras can capture detailed images, which is great for conference calls and selfies. However, if compromised, those multi-million-pixel arrays can benefit cyber-attackers too. But even with cameras covered, adversaries may still be able to glimpse at what users are up to based on information gathered from ambient light sensors, according to experts.

What are ambient light sensors?

The ambient light sensor is an important component inside not just smartphones, but a wide range of electronic devices, and can affect more than just screen brightness. Ambient light sensors can be extremely small – with a footprint measuring just L2.0 mm x W1.0 mm x H0.5 mm, in the case of OSRAM’s TSL2521 package, which is designed to fit between the display screen and the chassis of a smartphone.

Sensors enable smartphone cameras to correct for banding and flicker-generated artifacts produced by artificial light sources such as LEDs. And it’s interesting to note that Apple reportedly added a rear-facing ambient light sensor to its hugely popular iPhone for the first time in 2022.

Besides mobile devices, ambient light sensors can be found in many monitors, TVs, and even some projectors – to enable automatic brightness adjustment. And, given the large number of all of these devices in use today, it’s not a stretch to say that ambient light sensors are everywhere.

Mobile security experts have made another observation too. “Unlike front cameras, accessing ambient light sensors does not require any user permissions, at least on Android devices,” comments Yang Liu – a researcher at MIT, US, investigating imaging privacy threats from an ambient light sensor.

What’s more, the sensors – which are often overlooked as a security concern – are always on when devices are active to provide continuous functionality.

Reverse engineering the scene

In their study, Liu and his colleagues show how a single ambient light sensor – which typically contains a handful of photodiodes capable of detecting visible and sometimes also infrared light – can indirectly enable imaging. And all becomes much clearer when you consider light bouncing off of the subject in front of the sensor.

What are your thoughts on privacy smartphones 🕵️📱? TechHQ's @JT_bluebird1 reviews the @MurenaCom 2, which ships with @e_mydata "deGoogled" mobile OS. 'Out of the box' report here 📰👉https://t.co/Q01BOWIEhS pic.twitter.com/uqEFXd2JOF

— TechHQ (@techhq) October 19, 2023

“We argue that the ambient light sensor can enable imaging if one uses the screen as a controllable active source of illumination displaying a known video sequence,” writes the team in its paper – published (open access) recently in the journal Science Advances. “The ambient light sensor measures the corresponding intensity variation of light reflected off or blocked by the scene.”

In such a privacy threat scenario, it’s possible to use an inversion algorithm to reconstruct objects that may be obstructing light paths from the device screen to the ambient light sensor. And in its work, the team shows how it’s possible to decipher a hand positioned in front of the screen – with sufficient resolution to resolve four fingers.

Eavesdropping touch gestures.

The security researchers then extended this approach to eavesdrop on touch gestures using the ambient light sensor on a 17.3-inch tablet. Images collected over 20-minute periods show one-finger slide, two-finger scroll, three-finger pinch, four-finger swipe, and five-finger rotate gestures.

Having highlighted how such a privacy issue could be realized, the authors of the paper turn their attention to mitigations. The first is to apply tighter permissions to limit access. Also, the precision and refresh rate of the ambient light sensor could be reduced, making image reconstruction more challenging, but without affecting user experience.

Product designers may wish to implement changes too, based on the group’s findings. “From the hardware side, the location of the ambient light sensor should not be directly facing the user,” recommend the MIT researchers. “It could be on the side of smart devices, which could break the direct interaction of the screen and the light sensor, thereby reducing the privacy risks.”

A side-facing ambient light sensor could be a mismatch for consumers who like to use a smartphone case (although the problem could likely be solved easily with a drill). However, given the privacy risks, obscuring the input and adjusting screen brightness manually may be the safest option. At least for those with big secrets to hide.