Voice AI platform, ElevenLabs, today launched a new foundational AI speech model as the company emerges from beta. The company said that the new AI model, named Eleven Multilingual v2, has the ability to accurately produce ‘emotionally rich’ AI audio in 28 languages.
Built through in-house research, ElevenLabs said that its latest AI speech model underwent an 18-month developmental phase. During this time, the company studied the intricacies of human speech, built new mechanisms for the model to comprehend context and express emotions in speech generation, as well as synthesize new, unique voices.
Previously only available in English, Polish, German, Spanish, French, Italian, Hindi and Portuguese, the model now supports Chinese, Korean, Dutch, Turkish, Swedish, Indonesian, Filipino, Japanese, Ukrainian, Greek, Czech, Finnish, Romanian, Danish, Bulgarian, Malay, Slovak, Croatian, Classic Arabic and Tamil.
ElevenLabs highlighted that the expanded language support will enable content creators to craft localized audio material aimed at global markets spanning Europe, Asia, and the Middle East.
To generate speech with the Eleven Multilingual v2, users can input text in any of the supported languages into the text-to-speech platform.
Simultaneously, whether employing a synthetic or cloned voice, the company explained that distinct vocal attributes of the speaker will remain consistent across all languages, including their original accent. Moreover, a single voice can be used to generate speech across the 28 supported languages.
“Our text-to-speech generation tools help level the playing field and bring top quality spoken audio capabilities to all the creators out there,” Mati Staniszewski, CEO and co-founder of ElevenLabs, said in a statement. “Those benefits now extend to multilingual applications across almost 30 languages. Eventually we hope to cover even more languages and voices with the help of AI, and eliminate the linguistic barriers to content.”
The roll-out of Eleven Multilingual v2 follows the public release of Professional Voice Cloning earlier this month. The offering allows users to generate an accurate digital replica of their voices. With the latest update, the tool will now enable users to directly translate their voice audio to any of the newly added languages.
Since its beta launch in January, ElevenLabs asserts it has amassed over 1 million registered users across creative, entertainment and publishing spaces. The company announced a successful $19 million Series A raise in June led by former GitHub CEO Nat Friedman, ex-Y Combinator partner Daniel Gross, and Andreessen Horowitz.
ElevenLabs also recently partnered with D-ID, the generative AI video content platform, to combine their generative AI tools.
Source: mPost
