Are Smartphone Cameras About to See the Light? Discover the New Tech That Promises Sharper Photos

When it comes to smartphone cameras, bigger is better. Larger image sensors and lenses have more light to work with, so they can resolve more details. That’s especially important, because the filters that create color images also block about 70 percent of incoming light.

These color filters—laid out as a grid of red, green, and blue over the image sensor’s pixels—have been around for decades. But new approaches promise to exploit the physics of light to create color images without blocking out so many photons. Three such paths to sharper images were presented at the 2023 IEEE International Electron Device Meeting (IEDM). Now, these methods are beginning to emerge from the laboratory stage.

Samsung, for instance, will provide the front camera for China-based Xiaomi’s new phone that uses Samsung’s nano-prism technology for improved low-light performance. The technology doesn’t replace color filters; it uses diffraction to collect more light in each color-specific pixel. This enhances light sensitivity by 25 percent, according to the company.

Meanwhile, two new startups have developed ways to capture color images without filters. An Imec spinoff called Eyeo this month announced that it has raised €15 million in seed funding. And PxE Holographic Imaging showcased technology that combines depth sensing and color imaging, in January at this year’s Consumer Electronics Show (CES).

Both PxE and Eyeo are compatible with CMOS sensors, the most common digital image sensor used in cameras today. “The CMOS sensor is a very mature and strong platform to build upon. You have it in every device today,” says PxE’s founder and CEO Yoav Berlatzky. But “everybody wants more photons reaching their CMOS sensors.”

Eyeo’s Filter-Free Color Camera

Eyeo aims to commercialize the research presented by Imec at IEDM in 2023 for applications in consumer electronics, security, and more. By removing the color filter, the startup’s image sensor is made three times as sensitive as traditional CMOS sensors. “It’s as if we are finally opening the eyes of an image sensor,” says Eyeo CEO Jereon Hoet.

The color splitters in Eyeo’s image sensor guide light of different wavelengths to the appropriate pixels.Eyeo

It works by sending light through vertical waveguides that split light based on wavelength, then steer the photons to the appropriate pixel. The waveguides act like a funnel, so those pixels can be shrunk down to less than 0.5 micrometers in width, about half the size of a typical smartphone pixel. The technology also better matches the color sensitivity of the human eye than today’s filter-based imagers, according to the Imec research.

The color splitting tech is designed to be made with the existing tools and processes already used in CMOS foundries. The challenge comes on the software side. Eyeo is now working to ensure the sensor is compatible with its potential customers’ systems, according to Hoet.

In terms of applications, Hoet says the benefit of Eyeo’s smaller, more sensitive image sensors is especially clear for smartphones. However, he expects the technology will first be adopted for other uses, such as security systems for low-light conditions or augmented reality devices that require ultracompact sensors.

PxE Brings 3D to CMOS

The basic idea behind PxE’s approach is similar. Both companies aim to imitate color filters without losing photons and “somehow get the colors in the right place on the right pixel” by bending light waves, Bertlatzky summarizes.

In this version of the photo above, red lines indicate an object is closer, while blue lines mean it’s farther away. PxE

PxE’s technology uses a layer of diffractive material it calls a “holocoder” to not only create color images but also to act as a depth sensor (hence the “holographic” part of the company name). When white light passes through the holocoder, it creates an interference pattern that is recorded by the sensor. PxE’s algorithms then use that pattern to reconstruct a virtual 3D image—a hologram. The interference pattern also encodes information about the wavelength of light, so color (and infrared) images can simultaneously be reconstructed.

Berlatzky says PxE’s hardware is “less exotic” than color splitters and other approaches that use specially engineered metasurfaces. Much of its power comes from the software. “The basis of the algorithm is the physics of light,” Berlatzky explains. “You can think of it as if we’re running it in reverse, from the CMOS sensor back out to the world, and reconstructing what the camera is actually seeing, in terms of depth and image.”

Like Eyeo, PxE’s image sensor could be used in a range of applications—particularly in those that already have separate depth and image sensors, such as cars and smartphones.