Nvidia Research is introducing a new AI model that largely automates the process of creating virtual worlds, making it easier for developers to populate games and VR experiences with a diverse array of 3D buildings, vehicles, characters and more. Trained using only 2D images, GET3D generates 3D shapes with high-fidelity textures and complex geometric details. GET3D can generate “a virtually unlimited number of 3D shapes based on the data it’s trained on,” according to Nvidia, which says the objects can be used in 3D representations of buildings or the great outdoors, in games or the metaverse.

Expanding on Nvidia’s Omniverse toolkit, GET3D creates objects with “high-fidelity textures and complex geometric details,” the company writes in a blog post.

From animals to architecture to all sorts of vehicles, GET3D is “democratizing AI-powered 3D content creation,” said Nvidia VP of AI research Sanja Fidler, who leads the Toronto-based AI lab that created the tool. “Its ability to instantly generate textured 3D shapes could be a game-changer for developers, helping them rapidly populate virtual worlds with varied and interesting objects.”

“GET3D can generate around 20 objects per second using a single GPU,” writes Engadget, noting “it took just two days to feed around 1 million images into GET3D using A100 Tensor Core GPUs.”

At GTC 2022 last week, Nvidia CEO Jensen Huang predicted AI will auto-populate the 3D imagery of the metaverse, writes VentureBeat, explaining “he believes that AI will make the first pass at creating the 3D objects that populate the vast virtual worlds of the metaverse — and then human creators will take over and refine them to their liking.” CNET excerpted highlights of Huang’s comments in a video.

Trained on a library of 2D animal images, GET3D created a zoo’s worth of creatures. Fed photographs of vehicles, it generated a fleet of cars, trucks, vans and buses. The more diverse the training set, “the more varied and detailed the output.”

Engadget notes that another Nvidia AI tool, StyleGAN-NADA, enables the application of “various styles to an object with text-based prompts.” Available on GitHub, the tool lets you “apply a burned-out look to a car, convert a model of a home into a haunted house or, as a video showing off the tech suggests, apply tiger stripes to any animal.”

The Nvidia GET3D research team says future versions “could be trained on real-world images instead of synthetic data,” Engadget reports, explaining that “it may also be possible to train the model on various types of 3D shapes at once, rather than having to focus on one object category at a given time.”