The Flamera, a prototype augmented reality headset from Meta Platforms, won the coveted Best in Show award at Siggraph 2023 earlier this month, drawing attention for its buggy “fly eyes” appearance as well as lens design that takes an unconventional approach to solving AR’s “passthrough” challenge. Since AR glasses are meant to work IRL, developers typically use external cameras to let the real world pass through. Meta abandons existing solutions for a novel approach it calls “lightfield passthrough” that pairs image sensors with apertures to control the incoming light.

“Light that would contribute an incorrect perspective is blocked, while light that provides an accurate perspective is allowed to reach the sensors,” explains IEEE Spectrum. (Researchers from Meta’s Reality Labs detail lightfield passthrough in a Siggraph paper.)

This avoids the visual displacement that can occur when exterior cameras are used, as with the Meta Quest Pro. Headsets such as the Meta Quest Pro and Apple’s upcoming Vision Pro have an opaque case and use exterior cameras to reproject surroundings after processing the image to provide the correct perspective, a solution that “can lead to visual distortion,” IEEE reports.

The Vision Pro “visual computer” uses 12 exterior cameras and “pairs them with Apple’s custom R1 chip, which operates in tandem with an Apple M2 chip (like that found in the company’s laptops)” for a “computationally correct” perspective “much like the approach Meta tried with the Quest Pro, but Apple cranks the power up to 11,” writes IEEE.

Others, including Magic Leap and Microsoft with its HoloLens instead use “a transparent display that leaves a user’s vision unobstructed” but have other issues including “display quality, field of view, and brightness,” notes IEEE.

Anshel Sag, VP and principal analyst at tech advisory Moor Insights & Strategy called it “definitely among the best quality [augmented reality] passthrough I’ve ever seen,” lauding the image quality, resolution and frame rate.

“There was no latency and no differences between what my eyes would see and where my hand was positioned, versus where my hand was really located,” ModiFace Director of Software Development Edgar Maucourant told the engineering trade publication.

Another Reality Labs headset prototype, the Butterscotch Varifocal, won Siggraph’s Audience Choice award, boasting “an incredibly high angular resolution of 56 pixels per degree,” according to TechRadar.

Related:
Meta Looks to Pitch Next-Gen Ray Ban Stories Glasses to Live Stream Creators, Social Media Today, 8/24/23