DeepL, a German company that gained a profile with online text translation, has released DeepL Voice, a B2B tool that translates to captions in real time. DeepL Voice debuts in two iterations: DeepL Voice for Meetings, which allows participants to speak in their preferred language while serving colleagues with captions, and DeepL Voice for Conversations, which works on mobile devices, facilitating in-person, one-on-one conversations “with customers, colleagues or anyone else, in the language that works best for them,” the company explains, noting that real-time voice translation offers specific challenges. Continue reading DeepL Voice Translates 33 Languages to Captions in Real Time

Panjaya is a AI startup that aims to disrupt the world of video dubbing with a way to generate “hyperrealistic” recreations of a person’s voice speaking a new language. The system also automatically modifies the imagery to match lip and other physical movements to match the new speech patterns. Called BodyTalk, the technique is the launch point for Panjaya as it emerges from the stealth in which it conducted its R&D the past three years, backed by $9.5 million from venture funds and angel backers. The startup describes BodyTalk as “AI dubbing that looks and feels as natural as the original.” Continue reading BodyTalk Dubs into 29 Languages with Facial Moves to Match

YouTube is going all in on generative AI with nine new generative features announced at the Made on YouTube creator event in New York. Google DeepMind’s AI video generation model, Veo, is coming to YouTube Shorts later this year, enabling “even more incredible video backgrounds, breathing life into concepts that were once impossible to visualize,” as well as six-second standalone AI segments that can be incorporated into short videos. “Imagine a BookTuber stepping into the pages of the classic novel ‘The Secret Garden,’” suggests YouTube Chief Product Officer Johanna Voolich in describing the new AI-powered features. Continue reading YouTube Unveils New AI-Powered Features at Creator Event

OpenAI has debuted a new text-to-voice generation platform called Voice Engine, available in limited access. Voice Engine can generate a synthetic voice from a 15-second clip of someone’s voice. The synthetic voice can then read a provided text, even translating to other languages. For now, only a handful of companies are using the tech under a strict usage policy as OpenAI grapples with the potential for misuse. “These small scale deployments are helping to inform our approach, safeguards, and thinking about how Voice Engine could be used for good across various industries,” OpenAI explained. Continue reading OpenAI Voice Cloning Tool Needs Only a 15-Second Sample

Milano-based Voiseed demonstrated its web-based Revoiceit platform at CES, pitched as the best way to manage synthetic voice actors, particularly ensuring that synthetic voices present realistic emotions. The company describes it as a cloud-based solution that uses “generative AI to infuse virtual voices with human emotions and prosody, creating highly expressive, lifelike audio experiences.” While Revoiceit’s most obvious feature is its Studio (imagine Adobe Audition devoted to second-by-second management of voices), it may well be the product’s forthcoming API that provides real value to developers of entertaining technology products. Continue reading CES: Voiseed Upgrades Its Platform for Expressive AI Voices

Alphabet is developing an AI tool that would let creators generate music in the voice of famous recording artists. Lyor Cohen, global head of music for Google and its YouTube subsidiary, has reportedly been in discussions with music labels for several months about obtaining the rights to use songs by major artists to train an AI model in this manner. The discussions continue, but not without raising concerns in the music business. Meanwhile, other AI tools are already generating new content, but not without facing some resistance. The use of artificial intelligence to generate creative works in the style of others is being hashed out in the courts. Continue reading Music Industry Considers Impact of AI as New Tools Emerge

Captions, which leverages AI to help its customers produce “studio quality videos directly from their mobile devices,” has launched a new app called Lipdub that automatically translates and dubs content into 28 languages. The free download lets user dub anyone “and experience familiar voices and faces in a suite of new languages.” Lipdub’s translations not only duplicate what the company says is “the subject’s exact voice,” but also syncs lip movements to match. It also incorporates dialects and idioms, with options like Gen Z and Texas slang. Continue reading Captions Debuts AI Lipdub with Translation and Gen Z Slang

Spotify is using AI to drive podcast language translation in what sounds like the podcaster’s own voice, which has obvious implications for film and television dubbing. Working with podcast notables including Dax Shepard, Monica Padman and Bill Simmons, Spotify used AI to mimic their voices in Spanish, French and German for several episodes. The proprietary Spotify technology uses OpenAI’s new text-to-speech voice-generation technology as well as its open-source Whisper speech recognition system, which transcribes spoken words into text. The result, Spotify says, is “more authentic” and “more personal and natural” than traditional dubbing. Continue reading Spotify Uses AI to Copy Host Voices for Podcast Translations

Following several months of tests, YouTube is launching is multi-language audio track feature worldwide, with popular vlogger MrBeast helping to promote the new feature’s benefits. MrBeast, who has over 135 million global subscribers, is hoping to attract new subscribers to his channel now that the most popular videos are dubbed into 11 different languages. The multi-language audio feature allows creators to dub new and existing videos. YouTube says more than 3,500 multi-language videos have been uploaded to the site in 40-plus languages since January of this year. Continue reading YouTube Introduces Multi-Language Audio Tracks Worldwide

Deepdub, the Tel Aviv-based tech firm specializing in automated language localization using AI, has raised $20 million in Series A funding led by New York-based global venture capital and private equity firm Insight Partners. The two-year-old Israeli company will use the funds to expand its global sales and delivery reach, bolster R&D and other platform enhancements. The investment follows a year of explosive growth for Deepdub, which in December announced a multi-series partnership with First Look Entertainment’s streaming service Topic to dub into English its catalog of international documentary and TV series. Continue reading Insight Partners Leads $20M Series A Funding for Deepdub AI

Auto-dubbing, which uses artificial intelligence to translate content into different languages, is a technology on which the global entertainment industry has increasingly come to rely in finding audiences among the planet’s 7.2 billion people, speaking more than 7,000 languages in roughly 200 countries. Companies like Flawless, Deepdub and Papercup use different approaches to offload to computers much of the labor required to fill that distribution pipeline. Another company, Spherex, emphasizes cultural awareness and the need for heightened sensitivity in pursuit of hits that travel across borders. Continue reading AI Is Still a Work in Progress When It Comes to Auto-Dubbing

Artificial intelligence and machine learning are poised to revolutionize the dubbing process for media content, optimizing it for a more natural effect as part of an emerging movement called “auto-dubbing.” AI has impacted the way U.S. audiences are experiencing the Netflix breakout “Squid Game” and other foreign content, as well as helping U.S. programming play better abroad. Its impact is in its nascency. Soon, replacing rubber-lip syndrome with AI-enhanced visuals that enable language translation at the click of a button may become the industry norm.  Continue reading AI-Powered Auto-Dubbing May Soon Become Industry Norm

Netflix launched its Post Technology Alliance logo program last week, with plans to certify post-production products and systems from manufacturers and vendors that meet the streaming service’s technical and content delivery specs. “The Post Technology Alliance will build a more seamless experience from production through post-production,” said Chris Fetner, Netflix director of post partnerships and integrations, noting that certified vendors are “committed to better interoperability and faster innovation cycles.” Adobe, Avid, Blackmagic Design, Canon, Panasonic, RED Digital Cinema and Sony are among those with products already certified. Continue reading Netflix’s New Logo Program Will Certify Post Production Tools

A team from Pixar talked about the massive amount of work the company puts into “localization” or “regionalization” of every film they make. “John Lasseter, from the early days, placed a high importance on creating content for international markets, without the disruptive experience of subtitles or language they don’t understand,” said Pixar executive Cynthia Lusk. “That’s been a priority for the studio since ‘Toy Story.’” Lusk said that they regard each movie as a snowflake, with a different approach from title to title. Continue reading HPA Tech Retreat: Pixar Team Talks Localization, ‘Inside Out’

Following news of record quarterly earnings, Netflix told Wall Street investors that it plans to expand its video service to 200 countries in the next two years. Netflix added 2.43 million international subscribers in the fourth quarter of 2014 after expanding to 50 countries and its stock jumped more than 20 percent. As part of its ambitious expansion plans, Netflix also wants to start producing 20 original scripted series per year, which could help the company become more successful in international markets. Continue reading Netflix Plans to Add Original Series, Expand to 200 Countries