The Canon Americas Lab exhibit at CES this year featured a demonstration of Canon USA’s Volumetric X Motion Capture system that creates videos viewable from any camera angle. The multi-camera system leverages 2D data, 3D volumetric data, and analytical tools for sports and entertainment applications. The basis of the system is the same as the Free Viewpoint video system — Canon’s CES 2023 headliner — which was used in an NBA pilot for Cleveland Cavaliers’ alternative game stream. We checked in with the project’s researchers for updates, including ESPN highlights on Meta’s Xtadium VR app, a new U.S. volumetric studio, and how AI was used in this technology.

The Volumetric X Motion Capture system essentially is a greenroom surrounded by about 100 video cameras. A special synchronization algorithm lets the cameras capture the entire space simultaneously. The data is then converted into a 3D model that can be manipulated to generate a video where the camera’s viewpoint can roam freely.

A desired background is also rendered. The outcome is similar to the notable VFX style featured in “The Matrix” but the camera can move on any view plane here.

The alternate Free Viewpoint video stream of Cleveland Cavalier games is available only on the Bally Sports Ohio app and marketed as C-View. A few are posted on YouTube. Last year’s women’s NCAA Final Four games were hosted at the Cavaliers’ Rocket Mortgage Fieldhouse stadium, where the 100-camera system is already installed. ESPN collaborated with Canon to produce immersive VR highlights of these games for view with Meta’s Xtadium VR app on Quest.

Canon is pushing this volumetric technology beyond sports and into film and video production. While the Canon Volumetric Video Studio in Kawasaki, Japan had been the only production space with the 100-cameras system, recently U.S. production studio ZeroSpace began a collaboration with Canon to install a volumetric capture stage at their Brooklyn, NY location.

Related:
CES: Canon and Sony Tools Provide a Virtual Take on Sports, ETCentric, 1/7/23