A compelling panel on audio technology during ETC’s Virtual Reality Summit at NAB emphasized audio’s importance to effective VR storytelling. Moderator Jacqueline Bosnjak, Q Department chief executive/partner, recalls how, when she saw “Catatonic,” a VR experience from Vrse, she immediately knew she had to work in virtual reality. She asked panelists about their approach to VR projects. “The thing we’re all exploring is bringing depth to the timeline,” said Felix & Paul Studios director of sound technology Jean-Pascal Beaudoin.

“We have a timeline-based VR and then experiences like ‘The Martian’,” he said. “We need a bridge between those two worlds. Cinematic VR isn’t a game or linear cinema. There are a lot more things you can explore and bring into the gaming engine.”

Dolby Labs director of e-media technologies Nicolas Tsingos agrees. “VR brings a lot more flexibility than film, and creators want that,” he said. “There’s a new way of thinking about sound — closer to what game platforms do, but retaining some of the cool linear amount of control.”

Source Sound president/chief executive Tim Gedemer noted that, “using game engines for VR, many of these experiences don’t have the same dependencies that a full game has.”

“We can claim a whole lot more resources for audio for interactive experiences in VR,” he said, adding that, in the 1990s however, the cinematic and gaming worlds were quite separate. “These worlds are literally converging in the center… High quality video playback and interactivity — that’s the challenge,” he said.

Another challenge, said Nokia Technologies head of formats and platforms Jyri Huopaniemi is playback. “It’s about optimizing the user experience, the rich full 3D playback,” he said. “The scalability and understanding those requirements, formats and so on is the Holy Grail. There’s a lot of interest in that and we’re nowhere near it yet.”

DTS R&D staff engineer Dave Corsello added, “there’s a lot to think about on the input side as well.” “To track a position, all these things have to be supported from the get-go, formatted and propagated throughout production,” he said. “It’s a complicated tree of things.”

“Virtualization and rendering methods are also important,” he added. “How you hear it when you’re mixing it, by the time it gets to the end user and his different head size and earphones, it can be extremely different. Production has to compensate for all those things.”

“What’s happening in our role as audio professionals is different,” said Gedemer “They have to work sound into the script. We’re selling ourselves as audio directors, involved from Day One — from script to final playback.” Tsingos said he’s looking for tools to help mixers in particular. “In film, the screen is the center of the story, but in VR, the screen is all around you,” he noted. “You have to spend a lot of time to position sound.”

Huopaniemi explained that, “people don’t want to build a completely new workflow until VR is actually here,” and Gedemer pointed out that tools are “a moving target.” “They’ll be completely different in two or three months,” he said. “You have to constantly educate yourself.”

Still, said Bosnjak, “it’s a great time to be in VR.” “There are so many handshakes across the table,” she said. “No one is a competitor, everyone is sharing information.”