| November 19, 2024
ElevenLabs, known for its AI-powered voice cloning and text-to-speech capabilities, has expanded into conversational AI. On November 18, 2024, the startup launched a new feature enabling developers to create custom conversational AI agents via its developer platform, offering a wide range of customization options.
A Comprehensive Platform for Conversational AI
With this release, developers can design AI agents tailored to specific needs by tweaking variables like tone of voice, response length, and system prompts. The platform supports top language models like Gemini, GPT, and Claude, and allows developers to fine-tune response creativity (via temperature settings) and token usage limits.
Here’s what users can do on the platform:
- Select a Template or Build from Scratch: Choose the agent’s primary language, initial message, and persona.
- Integrate a Knowledge Base: Add files, URLs, or text blocks to train the agent on custom data.
- Fine-Tune Voice Features: Adjust latency, stability, authentication, and conversation length.
- Leverage Multiple SDKs: ElevenLabs’ SDK supports popular languages and frameworks like Python, JavaScript, React, and Swift.
For enhanced functionality, the platform also includes a WebSocket API for deep customization and the option to integrate proprietary language models.
Solving Pain Points in Conversational AI
Sam Sklar, ElevenLabs’ Head of Growth, revealed that their existing clients were already experimenting with conversational AI. However, challenges like integrating a knowledge base and handling customer interruptions prompted the company to develop a complete pipeline for conversational bots.
The new toolset makes it easier for companies to implement robust AI agents capable of maintaining realistic and responsive conversations.
Business Applications and Future Potential
Companies can define specific criteria for data collection, such as capturing customer names or emails during interactions. ElevenLabs also enables businesses to set natural language evaluation criteria to measure the success of interactions.
Although ElevenLabs currently lacks a standalone speech-to-text API, its existing text-to-speech pipeline underpins this new product. Future updates may include such an API, potentially challenging competitors like Google, Microsoft, Amazon, OpenAI’s Whisper, and AssemblyAI.
Competing in a Crowded Market
In the competitive conversational AI space, ElevenLabs faces rivals like OpenAI, Vapi, and Retell. However, ElevenLabs is betting on its ability to switch between models and deep customization as key differentiators.
With plans to raise funding at a valuation exceeding $3 billion, ElevenLabs is solidifying its position as a leader in voice AI and conversational technologies.
Explore ElevenLabs’ new capabilities on their developer platform and experience the future of conversational AI.
