At the HPA Tech Retreat in Palm Desert this week, Sony chief technology officer Don Eklund described how Sony has been using artificial intelligence as a toolset to create applications specific to its needs. “I was aware of AI but didn’t pay attention,” he said. “It’s now become pervasive.” He brought together three companies — Adobe, Rival Theory and Video Gorillas — that are researching and developing AI-enabled solutions over many years. Some of these tools are commercially available or will be soon.

Video Gorillas chief executive/managing partner Jason Brahms reported that his company is involved in a wide variety of AI-powered tools, including contextual metadata that is in use to support compliance logging. Another area is leveraging game engines to create photorealistic imagery using ray-tracing to replicate lighting environments.

With semantic segmentation and part maps, the system has been trained to know where people are in the frame and more granular information about those people. Video Gorillas has also worked on 2D-to-3D conversion with automated hole filling.

Brahms showed a synthetic map, and how AI uses it to create a photoreal version, via a generation of high-resolution images from semantic apps with interactive editing. He also showed the generation of images based on a text description. The breakthrough in AI has been GANs (generative adversarial networks), which is what Video Gorillas uses in its Bigfoot tool. GANs is composed of two opposing networks, the generator network and the discriminator network, that helps photoreal results improve with iterations.

“What does this mean for the M&E industry?” asked Brahms. “Right now, it’s a solution in search of a problem. We have lots of capabilities but no way of measuring results. It’s still early days, and the R&D products we’ve been focused on for the last five years are the foundational building blocks for our latest product releases and new R&D initiatives.”

Adobe research scientist Dr. Richard Zhang also identified GANs as a “watershed” moment for AI applications. He highlighted his research work on colorization enabled by AI, which predicts and applies colors to grayscale images, which can then be tweaked by humans. Zhang noted how media manipulation has been democratized. “Now any kid in a basement can put Nicolas Cage’s face anywhere,” he said.

The solution, he noted, is to also democratize image forensic tools. To that end, he showed a tool he’s working on that can spot composites and facial warps.

Rival Theory chief executive Kenny Lauer described the entertainment potential of the multiverse. His company built a deeply immersive interactive VR character, capturing performance and facial data of actor Becky, and then using that data to animate the digital persona Elise.” AI personas are based on real people, powered by cognitive and behavioral models,” he said. “They have memories, behaviors, emotions and interactions of their human counterparts.”

Self-help guru Tony Robbins, who is an investor in the company, will be the first AI persona the company will launch, but Lauer believes the technology can and will scale widely.