ElevenLabs has emerged from beta with its AI-driven voice synthesis platform, supporting over 30 languages. The company’s new in-house AI model identifies languages, generating emotionally rich speech in various tongues. The tool’s language capabilities aid voice cloning across languages, eliminating the need for text input. ElevenLabs aims to democratize content creation and is addressing concerns about misuse by introducing safeguards. It also envisions expanding into voice dubbing and cross-lingual emotional transfer.
Today marks a significant milestone for ElevenLabs, the AI-driven platform that has taken the internet by storm with its synthetic voice creation capabilities. The platform has officially transitioned out of its beta phase, boasting expanded language support across over 30 languages.
The company has harnessed a newly developed, in-house AI model to empower its tools. This advanced technology now enables ElevenLabs’ tools to automatically detect languages, including but not limited to Korean, Dutch, and Vietnamese. The result is the production of speech that is imbued with rich emotional nuances in these languages.
With this latest AI model enhancement, ElevenLabs’ users can harness the power of the platform’s voice cloning tool to articulate their thoughts and ideas across nearly 30 languages without the initial necessity of manual text input.
Mati Staniszewski, CEO and co-founder of ElevenLabs expressed his vision: “ElevenLabs was founded with the ambition of enabling universal access to content in all languages and voices. This release brings us a step closer to realizing this vision, making high-quality AI-generated voices accessible across diverse dialects. Our text-to-speech generation tools are instrumental in creating a level playing field, extending top-notch spoken audio capabilities to content creators worldwide.”
The inception of ElevenLabs was sparked by the less-than-satisfactory dubbing of American movies that Staniszewski and co-founder Piotr Dabkowski encountered during their upbringing in Poland. This led them to conceptualize a platform that could elevate this experience using AI technology.
After its beta launch in late January, ElevenLabs gained momentum swiftly, attributed to the platform’s impressive voice quality and generous free offering. However, the platform’s journey has seen its share of challenges, including instances of misuse.
Negative attention arose when the notorious online forum 4chan leveraged ElevenLabs’ tools to disseminate offensive content impersonating well-known figures such as actor Emma Watson. Additionally, The Verge journalist James Vincent highlighted how the platform was used to replicate individuals’ voices within seconds, generating audio clips that ranged from violent threats to expressions of discrimination.
In response, ElevenLabs pledged to implement new safeguards, such as restricting voice cloning to paid accounts and incorporating an AI detection mechanism.
Yet, ElevenLabs faces a broader debate in the industry concerning the impact on voice actors. As AI-driven synthesis gains prominence, voice actors are being asked to relinquish rights to their voices, raising concerns about potential displacement. Even major players like Activision Blizzard are delving into AI-assisted “voice cloning.”
ElevenLabs seems to view this trend as a natural evolution, collaborating with prominent names like Storytel, TheSoul Publishing, and MNTN for audiobooks, radio content, and video games. The company, backed by investments including $19 million from Andreessen Horowitz and DeepMind’s co-founder Mustafa Suleyman, aims to extend its AI capabilities to voice dubbing, creating a foundation for cross-lingual emotional and intonational transfer.
Looking ahead, ElevenLabs envisions introducing a mechanism for users to share voices on the platform, although specifics are yet to be unveiled.
