Although using AI and machine learning tools in production may remain a lofty goal for some, such tools are already in use in some video workflows, from dailies through mastering. Moderated by Netflix coordinator, production technologies Kylee Peña, a panel discussion described the tools available and how they’re being used in real world applications. Google senior cloud solutions architect Adrian Graham described his company’s now-open sourced TensorFlow technology, and how it’s being used by the M&E industry.

“A model can ingest and understand contents of an image or video file,” Graham explained. “Where Google comes in, you can leverage our APIs against your own library, and extend them based on your own content.” Peña noted that Deluxe is doing just that, using TensorFlow technology, and adding its own logic on top of that, for its newly launched Deluxe One solution.

“Deluxe One unifies all the stages of the lifetime cycle of a media asset, connecting contributors to distributors, “ said Deluxe Technology vice president of product development Weyron Henriques. “Within One, we built Synapse for the content creation side. We’ve modified and trained TensorFlow for specific purposes, allowing us to go all the way from dailies, or on-set if possible, to index the content for object recognition, color, environment, faces, actions.”

One use case, he added, would be to enable a VFX supervisor and editor to quickly find shots and assemble a playlist, cutting the time spent from days to minutes.

At HBO, said senior vice president, digital production services Barbara Ford Grant, data from dailies to masters is only part of the story. “Then we have an enormous amount of data that sits outside dailies — from VFX to scripts and call sheets,” she said. “We want to build ways to better utilize that.”

The goal, said Graham, is to “understand the patterns that exist throughout the production pipeline,” such as capturing camera information. “How we understand the data at any point in the pipeline is what I’m focused on,” he said.

Standards may not be the answer moving forward, said panelists. “If you’re a corporation and could run a model on customer sales and get a kernel of information, that’s a really hard value — and you don’t need standardization to understand that,” said Graham. For Ford Grant, on the production side, the need for analysis of schedule, budget and resources is “pretty universal,” although [content creators] all describe the parameters differently.

What is interesting to content creators is to be able to train a model to understand and predict resource usage throughout the production of a film. Ford Grant related the story of an engineer at Sony Picture Imageworks who was able to increase use of the render farm from 30 to 50 percent to 80 to 90 percent.

“Why not let an algorithm solve the problem,” queried Graham. “Yes, that’s what I’m looking for,” said Ford Grant. “Standards should be for things that everyone does. AI is just becoming democratized now, so the challenge is to create your own models.”