Apple has taken a novel approach to animation with Keyframer, using large language models to add motion to static images through natural language prompts. “The application of LLMs to animation is underexplored,” Apple researchers say in a paper that describes Keyframer as an “animation prototyping tool.” Based on input from animators and engineers, Keyframer lets users refine their work through “a combination of prompting and direct editing,” the paper explains. The LLM can generate CSS animation code. Users can also use natural language to request design variations. Continue reading Apple’s Keyframer AI Tool Uses LLMs to Prototype Animation

Impact and opportunity surfaced as the dominant theme of a full day of Digital Hollywood sessions devoted to artificial intelligence at CES 2024. We are in a period of disruption similar to the early 90s when the Internet went mainstream, said Forbes columnist Charlie Fink, moderating a panel of industry leaders from CAA, Paramount, HTC, Nvidia and Google. Yet despite the transformation already underway, panelists agreed that this is neither the first nor last technology to shift the status quo, more the latest example of inevitable change and adjustment. The current conversations around AI at CES are a refreshing departure after a few years of evolutionary, not revolutionary tech confabs. Continue reading CES: Digital Hollywood Session Explores AI at Inflection Point

Google has unveiled a new large language model designed to advance video generation. VideoPoet is capable of text-to-video, image-to-video, video stylization, video inpainting and outpainting, and video-to-audio. “The leading video generation models are almost exclusively diffusion-based,” Google says, citing Imagen Video as an example. Google finds this counter intuitive, since “LLMs are widely recognized as the de facto standard due to their exceptional learning capabilities across various modalities.” VideoPoet eschews the diffusion approach of relying on separately trained tasks in favor of integrating many video generation capabilities in a single LLM. Continue reading VideoPoet: Google Launches a Multimodal AI Video Generator

Stability AI has opened research preview on its first foundation model for generative video, Stable Video Diffusion, offering text-to-video and image-to-video. Based on the company’s Stable Diffusion text-to-image model, the new open-source model generates video by animating existing still frames, including “multi-view synthesis.” While the company plans to enhance and extend the model’s capabilities, it currently comes in two versions: SVD, which transforms stills into 576×1024 videos of 14 frames, and SVD-XT that generates up to 24 frames — each at between three and 30 frames per second. Continue reading Stability Introduces GenAI Video Model: Stable Video Diffusion

Having made the leap from image generation to video generation over the course of a few months in 2022, Meta Platforms introduces Emu, its first visual foundational model, along with Emu Video and Emu Edit, positioned as milestones in the trek to AI moviemaking. Emu uses just two diffusion models to generate 512×512 four-second long videos at 16 frames per second, Meta said, comparing that to 2022’s Make-A-Video, which requires a “cascade” of five models. Internal research found Emu video generations were “strongly preferred” over the Make-A-Video model based on quality (96 percent) and prompt fidelity (85 percent). Continue reading Meta Touts Its Emu Foundational Model for Video and Editing

Unity has officially released its Muse AI platform for general use in early access. Muse is a suite of AI-powered tools that streamline game development. The Muse package includes Muse Chat to source answers and generate code, Muse Sprite for 2D sprites generation, and Muse Texture, providing 2D and 3D ready textures. Originally announced in July, Muse is now offered at a $30 per month subscription. Also announced at the firm’s annual Unite conference was the next major software update, Unity 6, for 2024, and the deployment of Unity Cloud to connect development tools across projects and pipelines. Continue reading Unity Opens Beta for Muse AI, Sets General Release for 2024

The entrepreneurs behind the Myspace social network and gaming company Jam City have shifted their focus to generative AI and web3 with a new venture, Plai Labs, a social platform that provides AI tools for collaboration and connectivity. Plai Labs has released a free text-to-video generator, PlaiDay, which will compete with other GenAI video tools from the likes of OpenAI (DALL-E 2), Google (Imagen), Meta Platforms (Make-A-Video) and Stable Diffusion. But PlaiDay hopes to set itself apart by offering the ability to personalize videos with selfie likenesses. Continue reading Social Startup Plai Labs Debuts Free Text-to-Video Generator

Kaiber, the AI-powered creative studio whose credits include music video collaborations with artists such as Kid Cudi and Linkin Park, has launched a mobile version of its creator tools designed to give musicians and graphic artists on-the-go access to its suite of GenAI tools offering text-to-video, image-to-video and video-to-video, “now with curated music to reimagine the music video creation process.” Users can select artist tracks to accompany visuals to build a music video “with as much or little AI collaboration as they wish.” Users can also upload their own music or audio and tap Kaiber for visuals. Continue reading Startup Kaiber Launches Mobile GenAI App for Music Videos

YouTube has introduced three dozen new features that do everything from improve mobile playback and search to expand creator tools. When the words “like and subscribe” are uttered by creators, the buttons will respond with animations. Bigger preview thumbnails and a “lock-screen” feature designed to prevent mishaps are also in the new toolset. The improvements build on the “new look and feel” YouTube debuted last year, adding “more modern design elements and features that helped our viewers feel more immersed,” the company says. Some of the improvements target smartphones and connected TVs, while others are for the web app. Continue reading YouTube Adds Dozens of Mobile Playback, Creator Controls

ETC@USC will host its 8th vETC virtual conference at SIGGRAPH 2023 in Los Angeles, August 8-10. The event – which highlights significant presentations of emerging technologies and their impact on the M&E industry – will explore how generative AI, machine learning, and other compelling new tools help simplify building 3D worlds and tackle today’s computer vision challenges. Three days of sessions will be recorded and posted on ETC’s YouTube channel. For those attending SIGGRAPH who may be interested in attending the sessions (located at Z by HP Booth 215), visit the program guide, which includes a full schedule and speaker bios. Continue reading ETC Will Host Sessions at SIGGRAPH Conference This Week

ETC@USC will host its 8th vETC virtual conference at SIGGRAPH 2023 in Los Angeles, August 8-10. The event – which highlights significant presentations of emerging technologies and their impact on the M&E industry – will explore how generative AI, machine learning, and other compelling new tools help simplify building 3D worlds and tackle today’s computer vision challenges. Three days of sessions will be recorded and posted on ETC’s YouTube channel. For those attending SIGGRAPH who may be interested in attending the sessions (located at Z by HP Booth 215), visit the program guide, which includes a full schedule and speaker bios. Continue reading ETC Will Host Sessions at SIGGRAPH Conference Next Week

Sony Electronics is launching its Mocopi mobile motion capture system in the United States. Using a dedicated smartphone app for iOS and Android, the wireless system enables full-body motion tracking, captured by six small, lightweight sensors. Sony has been marketing Mocopi in Japan where virtual streamers (also called “VTubers”) have been using the system to drive avatars and fictional animated characters. Mocopi allows users to go mobile with virtual reality, loosening time and location constraints. Sony is now taking preorders for the $499 Mocopi system, which ships July 14. Continue reading Sony Offers Affordable Phone-Based MoCap System in U.S.

Epic Games is releasing MetaHuman Animator, which lets developers create nuanced facial animation by capturing an actor’s performance using an iPhone or stereo head-mounted camera system and a PC. The system eliminates the need for manual touch-ups, according to Epic, capturing “every subtle expression, look, and emotion” and replicating it onto a digital character for a faster performance capture workflow that allows more creative control. The new feature set uses a 4D solver to combine video and depth data with a MetaHuman representation of the performer. The animation is produced locally using GPU hardware, providing final results in “minutes.” Continue reading New Tool from Epic Simplifies High-Fidelity Facial Animation

Google-backed AI startup Runway has released Gen-2, an early entry among commercially available text-to-video models. Previously waitlisted in limited release, the commercial availability is impactful, since text-to-video is predicted as the next big bump in artificial intelligence, following the explosion of AI use generating text and images. While Runway’s solution may not be ready to serve as a professional video tool, this is the next step in development of tech expected to impact media and entertainment. Filmmaker Joe Russo recently predicted that within the next two years, AI may have the ability to create feature films. Continue reading Runway Makes Next Advance in Consumer Text-to-Video AI

Family history platform MyHeritage is releasing a mobile app called Reimagine that enables high-speed scanning of entire album pages to complement the company’s AI tools for restoring — and even facially animating — historical photos. Now users can easily import printed photos stored in albums by snapping page pictures on their iOS or Android device. The app will separate the individual photos, cropping and saving them as standalone images to which metadata can be added for indexing. The app also works with individual photos, or digital uploads from a camera roll. Continue reading Photo App Reimagine Brings Old Images to Life with AI Tools