As part of what it calls “production microservices,” Nvidia is adding an Avatar Cloud Engine (ACE) that lets game developers, as well as those who make tools and middleware, to integrate generative AI models into the digital avatars created for games and applications. The new ACE microservices “let developers build interactive avatars using AI models such as Nvidia Omniverse Audio2Face (A2F), which creates expressive facial animations from audio sources, and Nvidia Riva automatic speech recognition (ASR), for building customizable multilingual speech and translation applications using generative AI,” Nvidia says.

In a newsroom post announcing the ACE microsoervices, Nvidia says developers including Charisma.AI, Convai, Inworld, miHoYo, NetEase Games, Ourpalm, Tencent and Ubisoft are already using it.

Convai, which has already demoed a convincing use of the tech, is now showcasing a further advance that “empowers users to engage NPCs dynamically, seeking real-time responses and actions,” VentureBeat writes, adding that “characters now execute complex tasks, fetch items, and initiate interactions with other NPCs,” also known as non-playable characters (more or less extras, or supporting cast).

“Even during idle moments in the game, Convai-enabled NPCs engage with each other, expanding the spectrum of their interactions and perceptions within the virtual world,” VentureBeat explains, linking the demo.

Put simply, “ACE tech combines speech-to-text recognition and text-to-speech responses with generative AI facial animation and automated character personas to spit out computer-created character interactions,” explains Polygon, reporting on a CES 2024 demo which showed how ACE works. It is designed to integrate with game engines like Unity and Unreal Engine.

“Nvidia ACE opens up new possibilities for game developers by populating their worlds with lifelike digital characters while removing the need for pre-scripted dialogue, delivering greater in-game immersion,” Keita Iida, VP of developer relations at Nvidia tells IGN India.

Meanwhile, a Tom’s Guide writer says “Nvidia AI NPCs freak me out but they might be the next big thing for video games,” enthusing over the fullness of the conversation, and realistic lip movements in the Convai-produced demo.