Stability AI has released Stable Video 3D, a generative video model based on the company’s foundation model Stable Video Diffusion. SV3D, as it’s called, comes in two versions. Both can generate and animate multi-view 3D meshes from a single image. The more advanced version also let users set “specified camera paths” for a “filmed” look to the video generation. “By adapting our Stable Video Diffusion image-to-video diffusion model with the addition of camera path conditioning, Stable Video 3D is able to generate multi-view videos of an object,” the company explains.
“Stable Video 3D is a valuable tool for generating 3D assets, especially within the gaming sector,” Stability AI lead researcher Varun Jampani tells VentureBeat, noting that it additionally “enables the production of 360-degree orbital videos, which are useful in e-commerce, providing a more immersive and interactive shopping experience.”
SV3D “introduces significant advancements in 3D generation, particularly in novel view synthesis (NVS),” Stable Diffusion writes in a blog post.
“Unlike previous approaches that often grapple with limited perspectives and inconsistencies in outputs, Stable Video 3D is able to deliver coherent views from any given angle with proficient generalization. This capability not only enhances pose-controllability, but also ensures consistent object appearance across multiple views, further improving critical aspects of realistic and accurate 3D generations.”
Tom’s Guide writes that the company seems to be “on a mission to make the Star Trek Holodeck a reality,” with its orbital views. StabilityAI founder and CEO Emad Mostaque wrote on X: “Quite a lot still to come, the holodeck components all coming together. Every pixel will be generated.”
SV3D leverages technology Stability AI pioneered in Stable Video Diffusion and its flagship Stable Diffusion, as well as the Zero123 3D image model it released late last year. Stable Video 3D was “trained on massive datasets like LAION and LVD,” PC Magazine explains, citing the research report. “SV3D was then further trained on a refined version of the ‘Objaverse’ dataset, which consists of a massive range of 3D objects.”
A technical paper offers more details and experimental comparisons.
“SV3D is now available for commercial use with a Stability AI Professional Membership ($20 per month for creators and developers with less than $1 million in annual revenue),” says VentureBeat, noting that “for non-commercial purposes, users can download the model weights from Hugging Face.”
No Comments Yet
You can be the first to comment!
Leave a comment
You must be logged in to post a comment.