Project Music GenAI Control, an experimental work from Adobe Research, is setting out to change how people create and edit custom audio and music. The prototype tool lets creators generate music from text prompts, “and then have fine-grained control to edit that audio for their precise needs,” according to Adobe. Designed to help create music for broadcasts, podcasts or other “audio that’s just the right mood, tone, and length,” it can generate music from text prompts like “powerful rock,” “happy dance” or “sad jazz,” says Adobe Research Senior Research Scientist Nicholas Bryan, a creator of the technology. Continue reading Adobe’s Prototype AI Tool Is a ‘Photoshop for Music-Making’

Spotify is using AI to drive podcast language translation in what sounds like the podcaster’s own voice, which has obvious implications for film and television dubbing. Working with podcast notables including Dax Shepard, Monica Padman and Bill Simmons, Spotify used AI to mimic their voices in Spanish, French and German for several episodes. The proprietary Spotify technology uses OpenAI’s new text-to-speech voice-generation technology as well as its open-source Whisper speech recognition system, which transcribes spoken words into text. The result, Spotify says, is “more authentic” and “more personal and natural” than traditional dubbing. Continue reading Spotify Uses AI to Copy Host Voices for Podcast Translations

Stability AI is launching Stable Audio, a music generation AI tool that uses latent diffusion to deliver what the company says is high-quality 44.1 kHz music for commercial use. Stable Audio uses a web-based interface to generate music from text prompts and duration. Because its latent diffusion model architecture has been conditioned on text metadata as well as audio file duration and start time, it defeats a problem common to diffusion for generative audio — producing cohesive musical segments as opposed to arbitrary sections of a song that start or end in the middle of a phrase. Continue reading Stability AI Develops ‘Stable Audio’ Generative Text-to-Music

Meta Platforms is releasing AudioCraft, a generative AI framework that creates “high-quality,” “realistic” audio and music from text prompts. AudioCraft consists of three models: MusicGen, AudioGen and EnCodec, all of which Meta announced it is open-sourcing. Released in June, MusicGen was trained on Meta-owned and licensed music, and generates music from text prompts, while AudioGen, which was trained on public domain samples, generates sound effects (like honking horns and barking dogs) from text prompts. The EnCodec decoder allows “higher quality music generation with fewer artifacts,” according to Meta. Continue reading Meta’s AudioCraft Turns Words into Music with Generative AI