Panjaya is a AI startup that aims to disrupt the world of video dubbing with a way to generate “hyperrealistic” recreations of a person’s voice speaking a new language. The system also automatically modifies the imagery to match lip and other physical movements to match the new speech patterns. Called BodyTalk, the technique is the launch point for Panjaya as it emerges from the stealth in which it conducted its R&D the past three years, backed by $9.5 million from venture funds and angel backers. The startup describes BodyTalk as “AI dubbing that looks and feels as natural as the original.”

“Diverse audiences are transforming content consumption,” and Panjaya.ai is addressing that market with “seamless, culturally authentic video translation,” the company explains in an announcement.

Panjaya suggests that BodyTalk’s attention to detail — “the natural way that people speak with breaths, pauses, gestures, and other body movements that emotionally resonate with viewers” — is something on which competing techniques do not focus.

“Human-in-the-loop adjustments mean customers remain in control of their content’s message, tone, and emotion,” Panjaya points out. Currently, BodyTalk can translate to 29 languages via a Web-based dashboard.

Panjaya was co-founded by Hilik Shani and Ariel Shalom, deep learning specialists who worked for the Israeli government through 2021, and now head the startup as general manager and CTO, respectively. About 18 months ago they brought in Guy Piekarz as CEO.

TechCrunch calls Piekarz “a notable name to have onboard,” having in 2013 sold Apple his own streaming and recommendation platform, Matcha, for between $10 million and $15 million. Piekarz then “stayed with Apple for nearly a decade building Apple TV and then its sports vertical.”

What sets Panjaya and BodyTalk apart from competition in the dubbing space is a proprietary lip-syncing engine, “ensuring that the speaker’s mouth movements match the new language in the most accurate way possible,” writes AutoGPT, noting BodyTalk is particularly effective “when dealing with multiple speakers, different video angles, and various business use cases.”

While the B2B space is Panjaya’s current focus, with TED an early customer for its globally distributed TED Talks, the company is clearly trying to expand its client base. Its press release notes “60 percent of YouTube’s audience comes from non-English speaking countries,” citing a finding that “82 percent of audiences engage more with culturally relevant videos.”

Panjaya is offering a “try for free” deal at panjaya.ai.

Related:
How Meta’s AI Dubbing Breaks Down Language Barriers, PRWeek, 9/30/24
D-ID Launches an AI Video Translation Tool That Includes Voice Cloning and Lip Sync, TechCrunch, 8/21/24
YouTube Announces Expansion of Auto-Dubbing to More Creators and Languages, Social Media Today, 9/19/24
Reddit Is Bringing AI-Powered, Automatic Translation to Dozens of New Countries, TechCrunch, 9/25/24