Quantum designs bury face biometrics under smartphone screens

Getting your Trinity Audio player ready...

High-end smartphones are works of art, packing many times the computing power needed to put humans on the moon into a sleek form factor topped with a pin-sharp high-resolution color display. However, some might argue that modern devices aren’t as beautiful as they once were. Bezel-less designs engineered to maximize screen size, force developers to place front-facing camera hardware and face biometrics into notches and pill-shaped cut-outs.

Apple’s designers have reconciled themselves, for the time being, to hiding the image sensor array – which features a regular camera, plus an infrared (IR) projector and camera – using cleverly designed graphics. Adjacent portions of the display are switched to black to mask the cutout sections of the screen – a feature dubbed ‘dynamic island’ as it includes app-specific information, which launched with the iPhone 14 Pro.

Prospects for under panel cameras and sensors

A more elegant solution would be to hide all of those components under the display, paving the way for full-screen notchless iPhones and other smart devices featuring face biometrics and front-facing cameras. But that’s easier said than done. Firstly, when it comes to cameras, burying the optics under the screen is a step backward in image quality, with less light hitting the sensor.

And things become particularly tricky when you examine how smartphones capture facial biometrics. Considering Apple’s Face ID, the iPhone’s front-facing image sensor array projects a pattern of structured light onto the face of the viewer and builds – using the built-in IR camera – a depth map of the target, which can be compared to a stored user profile.

The issue, as Jacky Qiu – VP of Strategy at OTI Lumionics – told TechHQ, concerns the display electrodes, which sandwich the smartphone screen’s light-emitting red, green, and blue pixels. “Metals are a good absorber of IR light,” he points out. “So if you put a camera behind the screen, the IR signal is attenuated.”

In fact, the attenuation factor can be as high as 2000, which doesn’t leave much IR information for the detector to work with. And so, based on that evidence, the prospects for burying face biometrics under smartphones don’t look promising. However, there are tiny windows of opportunity – literally.

Display makers are using self-assembling molecules produced by OTI Lumionics – a technology that the Canadian firm dubs CPM patterning – to engineer IR transmission through the display electrodes. “You open up many small windows in the range of a few microns,” said Qiu.

OTI Lumionics, which was spun out of the University of Toronto in 2011, has been busy finding a killer application for its advanced materials technology, exploring a variety of use cases along the way. “You need to find one or two application spaces to get a footing and then climb from there,” Qiu, a member of OTI’s co-founding team, explains.

There are countless examples of bright ideas that shine in the lab, but struggle to find their niche in the market. And Qiu advises tech start-ups to be conscious of the bigger manufacturing picture. “Always be in touch with the end consumers,” he said. “And consider their roadmaps and their needs.”

Endorsement from display makers

In 2022, OTI Lumionics announced that it had raised $55 million in series B funding to enable under display cameras and sensors. And the investor list includes venture arms of big-name display makers such as LG Technology, Samsung, and United Display Corporation (UDC).

It’s worth adding that while the focus may be on enabling full-screen high-end smartphones thanks to under panel front-facing cameras and face biometrics, OTI’s patterning technology can serve other industry sectors too. The firm lists semiconductor, energy, battery electrodes, meta materials, catalysts, communications, and medical devices as other applicable markets beyond displays.

And helping the company in its search to find the right molecule has been the rise in quantum computing methods. Quantum computing holds great promise for being able to simulate materials behavior. And while current machines lack sufficient numbers of qubits to represent the long molecules used in display electrode patterning, it turns out that running quantum-inspired algorithms on classical machines ended up playing a vital role in materials discovery.

Display manufacturing is a huge endeavor, involving pattering large sheets of glass the size of an automobile, which are then cut down to device sizes. And trial and error is too expensive when it comes to getting materials approved for manufacturing. Suppliers such as OTI need to narrow down the number of candidates to a manageable number.

Artificial intelligence methods have merits, but only when fed with large amounts of data. However, that takes developers back to the problem of how to generate sample information affordably, which is where synthetic data fit in. And running quantum-inspired algorithms on classical computers has provided a major boost in generating synthetic data that’s a level above relying purely on density functional theory (DFT) – a common route to calculating the electronic structure and properties of molecules.

Given the progress being made by OTI Lumionics, its display partners, and other stakeholders in the smartphone supply chain, the big question is when will we see high-end smartphones with truly full-screen displays?

Apple has sent out invitations this week for its latest event on 12 September 2023, when many expect the tech giant to announce its new flagship iPhone 15 lineup. But considering the views of display industry analysts, it appears unlikely that consumers will see under panel Face ID until late 2024 for Pro models, and 2025 for standard iPhones.

Based on the technical challenges remaining for under panel cameras to meet discerning brands quality requirements as well as panel manufacturers' cost requirements, I still believe this roadmap makes sense for the iPhone. pic.twitter.com/3ck5X3sVcL

— Ross Young (@DSCCRoss) May 10, 2022

And waiting to purchase a new mobile device is a good way for consumers to conserve materials and limit their environmental footprint, as smartphone circular economy studies have shown.

However, users should factor in the security benefits of using face biometrics. And if your current smartphone doesn’t have a feature such as Face ID or an equivalent, then it may be worth considering making a purchase – which could be of a refurbished model.

In the interview with TechHQ, Qiu points out that IR-based face biometrics have advantages over simply using images in the visual spectrum. Some mobile users post pictures of themselves to social media every day, and this is just one source that could be exploited to undermine the security of regular camera-based authentication systems, combined with generative AI techniques capable of creating ever more believable deepfakes.

Liveness testing using regular images is being threatened by developments in generative AI.

Apple puts the odds of a random person in the population having a face that can unlock another user’s iPhone at 1 in a million. However, the probability increases for twins and siblings that look alike. Device makers are always pushing the envelope, however.

Last year, Samsung Display revealed that it’s working on technology that sits underneath an OLED screen, which can read multiple fingerprints simultaneously (IMID 2022 keynote address on YouTube). And the screen manufacturer believes that using three fingerprints simultaneously for authentication is billions of times more secure than relying on the impression gathered from a single digit.

The topic of digital id’s is growing. Smartphones are used to authenticate, using face biometrics and other methods, to various services. And while some may be concerned about putting passport details, driving licenses, and other PII in the digital domain, there are plus points for consumers.

For example, in the future users may be able to share just the necessary portions of their ID, such as whether they meet age requirements, rather than having to hand over their address and other personal information as well, which may be irrelevant to a particular ID check.