Israeli startup PxE Holographic Imaging has developed a drop-in replacement sensor for any camera that holographically captures depth information without lidar or other hardware. Or more specifically, it augments any existing sensor with this capability, so any existing sensor OEM’s product can be adapted. Imagine face ID without an IR projector and sensor, your videoconference camera able to send a 3D image, or volumetric capture suddenly becoming more affordable. Extraordinarily, the physics appears to check out, and PxE demonstrated the technology to us at short- and room-size range in their CES suite at The Venetian Las Vegas.

Light is both wave and particle, and existing camera sensors are particle assessors. Crudely put, they read the ‘amount’ of red, green and blue in photons hitting a sensor pixel.

What PxE claims to do is allow existing sensor hardware to assess light as a wave. They replace a sensor’s Bayer filter with a transparent diffractive-filter array (DFA) that creates interference patterns across the surface area of a sensor. These interference patterns contain information that can be computationally decoded on device.

DFAs have been known to science for a decade as a method to capture RGB information using a monochrome sensor (which PxE can also enable). PxE appears to have an algorithmic go-round to extract additional scene information.

In fact, PxE’s claims go further. The company says existing sensor technology can now capture IR, images can be digitally de-blurred, complete camera modules can drop elements to reduce size, and sensors automatically gain highly significant improvements in low-light performance.

We can say for sure we saw a single-sensor webcam-class camera create a 3D rig of a face in real time which the user could manipulate, and on a separate rig, display the user’s face in 3D on a 3D display. Just this application presents fascinating possibilities for XR gaming, cameras in wearables like glasses, and simple volumetric capture.

Where’s the catch? PxE claim there is none, not in gamut or effective resolution. The physics appear to support that on the capture side; however, we did see some significant lag between capture and display. Decoding is clearly testing the limit of consumer silicon and we wonder if lossy compromises like bit reduction, quantization or dithering are necessary. Either way, that would probably be solved by silicon horsepower eventually.

A promotional video of PxE’s Holographic RGB-IR-Depth camera is available on the company’s CES press release.