The world’s first AI-powered movie camera has surfaced. Still in development, it aims to enable filmmakers to turn footage into AI imagery in real time while shooting. Called the CMR-M1, for camera model 1, it is the product of creative tech agency SpecialGuestX and media firm 1stAveMachine, with the goal of providing creatives with a familiar interface for AI imagemaking. It was inspired by the Cine-Kodak device, the first portable 16mm camera. “We designed a camera that serves as a physical interface to AI models,” said Miguel Espada, co-founder and executive creative technologist at SpecialGuestX, a company that does not think directors will use AI sitting at a keyboard.

The CMR-M1 records real images and transforms them with a generative AI video-to-video model, the partners explain in an announcement. The camera incorporates a FLIR sensor, a Snapdragon CPU, and a viewport. The capture is 1,368 x 768 at 12 frames per second, with the AI processing taking place in the cloud.

“The real images are uploaded into a cloud server where a Stable Diffusion workflow is executed,” the companies explain, noting that the prototype doesn’t execute in real time.

“There is a latency between the recording and the transformation, but new AI models such as StreamDiffusion will allow real-time processing,” the announcement notes, referring to a new technology led by UC Berkeley computer scientists described on its GitHub page as “a pipeline-level solution for real-time interactive generation.”

“The new AI models for video creation models, such as Sora, Stable Diffusion, KREA, Pika, Runway or Kling are reaching a level of realism that will completely change the film industry,” according to the CMR-M1 announcement. As a video-to-video technology, the new camera wants to bring things full circle, keeping filmmaking part of the process.

“The developers certainly see AI generation as a tool for filmmakers to use much like tripods and lighting, and the CMR-M1 is very much a way for AI to break out of the office and capture real-world scenes,” writes Digital Camera World, explaining that “by utilizing an NFC chip with an identifier the user can personalize their styles and workflow, in addition to creating a model that can be trained on their own personal video and images.”

Imagine, Digital Camera World posits, “being able to capture pre-post-processed video in real time, or capturing live-action footage and it being instantly turned into animation.”

The CMR-M1 prototype “comes equipped with five Stable Diffusion LoRAs, including a ‘Blooming Nature’ style that turns images into a colorful jungle and an ‘Old Money’ style that creates classic luxury scenes with tuxedos, gold coins and rugs,” reports Ad Age.

Related:
The First AI Cinema Camera Is Wild (Video), Theoretically Media, 6/21/24
How Will AI Change the Advertising Film Industry?, Forbes, 6/21/24