Eleven Realtime Review: Can One Voice Model Make AI Agents Feel More Natural?
Voice agents have traditionally been assembled from separate components: speech recognition to understand the caller, an LLM to decide what to say, turn-taking logic to determine when to respond, and text-to-speech to produce the voice. Each part can work well on its own and still create an awkward conversation when the handoffs are slow or poorly coordinated.
Eleven Realtime is ElevenLabs' new speech-to-speech model family designed to reduce that integration work. Announced on October 20, 2026, it launches with two options: Realtime v1 Mini for low-cost, natural conversation and Realtime v1 Max for complex instructions, reasoning and multi-step tool use. Both support more than 90 languages and can use voices from the ElevenLabs library, which the company says contains more than 17,000 voices.
The important distinction is that Eleven Realtime is not simply another text-to-speech model. It is a complete real-time voice-agent option, with speech input, response generation and speech output coordinated as one managed system. It targets voice assistants, customer support agents, tutors, interactive characters and phone-based workflows.
The first public benchmark results show a clear split between Mini and Max. Max reaches 98.8% on Big Bench Audio and completes 87.1% of tool-use tasks, while Mini scores 75.9% and 60.3% respectively. Mini, however, leads slightly on Full-Duplex-Bench, which tests conversational dynamics such as pauses, interruptions and turn-taking.
That makes the product interesting for a practical reason: there is no single best voice-agent model for every use case. The better choice depends on whether your priority is cost, conversational flow, spoken reasoning, tool reliability or deployment constraints.
QUICK ANSWER
Eleven Realtime is ElevenLabs' real-time speech-to-speech model family for building voice agents without separately configuring every speech recognition, LLM, turn-taking and text-to-speech component. The October 20, 2026 launch includes Realtime v1 Mini for low-cost conversation and Realtime v1 Max for complex instructions and multi-step tool use.
Start with Mini for most conversational products where cost and responsiveness matter. Choose Max when the agent must reason through complex requests, follow long instructions or call tools reliably. Max costs more per token, and its current availability has restrictions: it is not available in India or Singapore data-residency environments, while Mini is available in both. Check live pricing and workspace availability before deployment.
1. What Is Eleven Realtime?
Eleven Realtime is a family of low-latency voice-agent models offered through ElevenLabs. Instead of requiring developers to choose and connect separate automatic speech recognition, language-model and text-to-speech components, Realtime provides a managed speech-to-speech experience designed around live conversation.
This matters because conversational quality depends on more than the words an LLM generates. A voice agent must recognize when someone has finished speaking, handle interruptions, decide whether a short pause means the user is thinking or done, respond quickly, and deliver audio that fits the moment. Weak handoffs between components can make a technically capable agent feel slow or robotic.
ElevenLabs offers two models at launch. Mini is the default option and prioritizes price and speed. Max reasons before responding and is aimed at more complex instructions and multi-step tool calls. The system prompt, voice, tools, knowledge base, language settings and conversation-flow settings remain configurable in both modes.
Voice agents are a specific type of AI agent: they interact through spoken language but may still need tools, memory, retrieval, permissions and escalation. For the foundations, read our What Is an AI Agent? guide, which explains the components that turn a language model into a system that can complete tasks.What Is an AI Agent? Beginner Guide With Examples (2026)
2. Eleven Realtime Specifications
Eleven Realtime is positioned as a complete managed voice-agent option rather than a standalone speech synthesizer. It combines the operational pieces needed for a live exchange and offers different model variants for different workloads. The public documentation identifies the model IDs and API integration path, while noting that the underlying component models are not disclosed.
One technical detail is worth keeping precise: despite the phrase speech-to-speech, ElevenLabs documents Mini and Max as using a cascaded architecture. The company has not publicly disclosed the specific underlying component models. It is therefore more accurate to describe Realtime as an end-to-end managed voice-agent system than to claim it is a single fused neural model that directly maps audio input to audio output.
3. Eleven Realtime Mini vs Max
Mini and Max share the same broad product interface, but the benchmark results suggest different operating points. Mini is built for natural conversation at lower cost, while Max is designed to reason before responding and handle more demanding workflows.
For most teams, Mini is the right first experiment because it is the default low-cost option and performs strongly on conversational dynamics. Move to Max when evaluation shows that Mini misunderstands complex spoken requests, misses tool steps or cannot reliably complete the workflow.
Avoid choosing Max just because its benchmark numbers are higher. If the agent only answers simple FAQs, schedules basic appointments or plays an interactive character, the extra reasoning capability may not translate into enough user value to justify higher usage costs.
4. Benchmarks: How Good Is Eleven Realtime?
ElevenLabs says Artificial Analysis independently evaluated both models on three public benchmarks: Big Bench Audio, Full-Duplex-Bench and Speech Agent Arena. These tests measure different parts of voice-agent quality, so the results should be read together rather than collapsed into a single score.
The biggest separation is Big Bench Audio. Max answered 98.8% of the benchmark's 1,000 voice-based reasoning questions correctly, compared with 75.9% for Mini. This suggests Max is the stronger choice when users ask questions that require careful interpretation and a correct spoken answer.
Mini has the higher Full-Duplex-Bench score, 95.8% versus 94.1%. That benchmark targets interaction mechanics rather than general reasoning. It is a reminder that an agent can be better at taking turns naturally without being the best at complex problem-solving.
Speech Agent Arena scores are 998 for Max and 988 for Mini in the reported snapshot. Those numbers are benchmark ratings, not percentages. They should not be interpreted as 99.8% or 98.8% user satisfaction.
These are useful public results, but benchmark scores do not guarantee the same performance in every product. A customer support agent using a narrow knowledge base, a multilingual tutor and a phone agent that updates a CRM all have different failure modes. Test with the languages, accents, interruptions, tools and business rules your users will actually encounter.
5. Latency: 570 to 620 Milliseconds to First Audio
ElevenLabs reports average time to first audio of 0.57 seconds for Max and 0.62 seconds for Mini on the published evaluation. That is a strong starting point for live voice applications because users notice silence more acutely in spoken interaction than in a text chat.
The figures are benchmark measurements, not a promise that every deployed call will start speaking within the same interval. Real-world latency depends on network conditions, audio capture, client device, server region, tool calls, prompt size, model reasoning and the time needed to decide whether the user has finished speaking.
The gap between Mini and Max is small in this particular measurement, with Max starting audio about 50 milliseconds sooner. That does not mean Max is universally faster: its main distinction is higher reasoning and tool-use performance, and complex tasks may take longer before the agent has a complete answer.
When evaluating a voice agent, measure time to first audio, time to first useful answer, interruption recovery and end-to-end task completion separately. A system can begin speaking quickly and still fail if it gives a partial answer, talks over the user or takes too long to finish an action.
6. Tool Calling and Agentic Workflows
Tool use is one of the clearest reasons to consider Max. In the published results, Max completed 87.1% of tool-use tasks, compared with 60.3% for Mini. This is a substantial gap for applications where the agent must do something rather than simply talk.
Examples include checking an order, looking up a customer record, booking an appointment, qualifying a lead, retrieving a policy, creating a support ticket or moving a conversation through a multi-step process. In these cases, a convincing voice is not enough. The agent must select the right tool, pass valid arguments, interpret the result and explain what happened.
A robust implementation should still validate every tool call. Use schemas, permission boundaries, allow-listed actions, idempotency for write operations, and human approval for consequential changes. Model-level task success does not remove the need for application-level safeguards.
For teams orchestrating several agents or tools, our LangGraph guide explains how to structure state, handoffs and execution. If your agent can access production systems, our security guide covers permissions, sandboxing and secrets.How to Use LangGraph for Multi-Agent Systems (2026) and How to Secure AI Coding Agents: Permissions, Sandboxing, MCP & Secrets
7. Voice Quality, Languages and Voice Library
ElevenLabs says both Realtime variants support more than 90 languages and can use any of the 17,000-plus voices in its library. That is useful for teams that want to deploy the same agent across regions while choosing different voices for brand, audience or language requirements.
Language coverage does not automatically mean identical performance across languages. Speech recognition accuracy, pronunciation, names, regional accents, code-switching and cultural phrasing can vary. Test the exact languages and dialects your customers use, including mixed-language requests and domain-specific vocabulary.
Voice selection also affects perceived quality. A support agent should be clear and calm; a tutor may need encouraging delivery; an interactive character may require expressive delivery. Choose voices through user testing rather than assuming the most realistic voice is always the best choice.
Because Realtime is designed around live interaction, test more than isolated sentences. Listen to long conversations, interruptions, repeated questions, emotional turns and the way the agent recovers after an error. The user's impression comes from the whole conversation, not a single audio sample.
For a deeper look at speech generation on the model side, our review of Gemini 3.8 Flash TTS and Flash-Lite TTS compares another approach to low-latency speech output.Gemini 3.8 Flash TTS and Flash-Lite TTS Review (2026)
8. Eleven Realtime Pricing: What You Need to Know
ElevenLabs documents Realtime billing as the standard ElevenAgents per-minute rate plus the model's input, cached-input and output token charges. Max costs more per token than Mini. The company directs developers to its live pricing pages for current rates, so this review does not assign a fixed per-token dollar figure that may become outdated.
This pricing structure means comparing only token prices can be misleading. A useful cost model includes call minutes, input and output usage, cached context, tool calls, retries, fallback behavior and the percentage of conversations that need human escalation.
Before choosing a model, run a pilot with realistic conversations and calculate cost per resolved task. If Max reduces failed tool calls or prevents expensive transfers to human agents, its higher token cost may be justified. If Mini resolves the same request reliably, paying more for Max may be unnecessary.
Track the following cost drivers separately: platform minutes, model input tokens, cached input, generated output, tool retries and human transfers. Also consider the engineering cost of maintaining a custom speech pipeline. A managed service can be worthwhile even when its per-call cost is not the lowest, if it reduces integration and maintenance work.
9. OpenAI Realtime API Compatibility
ElevenLabs says its Realtime API follows the same wire protocol as OpenAI's Realtime API, including client and server events over WebSocket. If you already have an application built around the OpenAI Realtime SDK, the documented integration path is to point the client at the ElevenLabs endpoint and update the session configuration and model name.
Compatibility can reduce migration effort, but it does not mean every application can switch providers without testing. Authentication, available models, voice configuration, audio format, tool definitions, session limits and platform-specific capabilities still need to be checked.
For browser-based clients, the documented approach is to request a short-lived client secret from your backend and pass it to the client. Do not expose a permanent server API key in browser JavaScript. On WebSocket, developers may need to handle PCM audio capture and playback directly; automatic audio wiring differs by transport.
This is a useful option for teams that already have a real-time voice interface and want to evaluate another provider without rebuilding the whole client protocol. Still run a small migration test before replacing a production endpoint.
10. Eleven Realtime vs a Custom Voice Stack
Realtime is not automatically better than a custom stack. A custom architecture remains appropriate when you must use a specific LLM, Amazon Bedrock, a custom speech engine, or workflow nodes that override managed settings. ElevenLabs itself recommends Custom mode for those cases.
The right comparison is the complete deployed system: quality, end-to-end latency, engineering time, failure rate, monitoring and cost. If the managed option gets you to a production-quality agent faster, that saved engineering effort is part of its value.
11. Realtime vs ElevenAgents Custom Mode
ElevenAgents offers two broad approaches. In Custom mode, developers select the LLM, text-to-speech model and automatic speech recognition settings. In Realtime mode, ElevenLabs manages those components as a coordinated model configuration.
Realtime mode manages the LLM, reasoning effort, thinking budget, parallel tool calls, cascade timeout, TTS model, expressive mode and ASR provider. The system prompt, voice, tools, knowledge base, languages, temperature and conversation-flow settings continue to apply.
A key constraint is that workflow nodes cannot override settings managed by Realtime mode. If your workflow relies on such overrides, keep it in Custom mode rather than forcing an incompatible configuration.
12. Privacy, Compliance and Regional Availability
The current ElevenLabs documentation lists different availability for Mini and Max. Mini is available in US, EU, India and Singapore data-residency environments. Max is available in the US and EU, but not in India or Singapore data-residency environments.
These are product availability rules, not a substitute for a compliance review. Organizations should confirm workspace configuration, contract, data-processing terms, retention settings and applicable legal requirements before sending personal, financial, medical or confidential data to a voice agent.
If your deployment requires India data residency, Max is not currently an option under the published rules. Mini may be available, but you still need to verify that its capabilities and compliance posture meet the requirements of your use case.
13. Best Use Cases for Eleven Realtime
A useful design pattern is to keep the voice agent's role narrow. Define which questions it can answer, which tools it can call, what evidence it must consult and when it should hand off to a human. A stronger model helps, but clear workflow boundaries make performance more consistent.
14. Limitations You Should Know
- Mini is weaker than Max on the published spoken-reasoning and tool-use benchmarks.
- Max costs more per token than Mini, so high-volume deployments need cost-per-resolution testing.
- Realtime models use a cascaded architecture; they are not documented as a single fused audio-to-audio model.
- The underlying component models are not publicly disclosed in the Realtime documentation.
- Realtime availability is rolling out gradually and may not appear in every workspace.
- Max is not available in India and Singapore data-residency environments under current documentation.
- Realtime mode manages several model settings, limiting custom component overrides.
- OpenAI Realtime API compatibility reduces migration effort but does not remove the need to test audio transport, authentication, tools and session behavior.
- Benchmark results do not guarantee performance for every language, accent, business domain or noisy environment.
- Actual cost includes platform minutes and token usage, not just a single model rate.
15. How to Test Eleven Realtime Before Production
Do not judge a voice agent only by a short demo. Build a test set that resembles the real conversations it will handle. Include ordinary requests, ambiguous requests, interruptions, background noise, varied speaking speeds, language switching, names and domain-specific vocabulary.
Test every tool with valid inputs, missing fields, permission failures and timeouts. Verify that the agent does not claim an action succeeded when the tool returned an error. Measure first-audio latency and time to complete the entire task, not just the speed of the first spoken word.
Compare Mini and Max on the same conversation set. Score factual correctness, turn-taking, interruption handling, voice naturalness, tool correctness, latency, cost per resolved conversation, retries, human transfers and user satisfaction. Keep the test set stable as prompts and models change so that improvements are measurable.
- Test normal requests, ambiguous requests and incomplete sentences.
- Interrupt the agent while it is speaking and check whether it yields naturally.
- Include background noise, different microphones and varied speaking speeds.
- Test language switching, names, addresses and domain-specific vocabulary.
- Measure time to first audio and time to completed task.
- Test tool errors, missing fields, permissions and timeouts.
- Track cost per resolved conversation, retries, human transfers and satisfaction.
- Run privacy, retention and data-residency checks before using sensitive data.
16. Recommended Implementation Workflow
A reliable deployment starts with a narrow use case and expands only after the agent passes realistic tests. Start with Mini unless the task clearly needs advanced reasoning or tool use. Write concise system instructions, define tone and escalation rules, connect only the tools needed for the task, and validate every tool response.
Use Max if Mini cannot reliably follow complex instructions or complete multi-step tools. Keep permanent API keys on the server and use short-lived client secrets for browser sessions. Monitor latency, task completion, cost and fallback behavior after deployment, and recheck pricing and data-residency constraints before scaling.
This approach makes model choice an evidence-based decision rather than a preference. The cheapest model that reliably completes the required task is usually the right default.
17. Is Eleven Realtime Worth It?
Yes, for teams that want to ship a natural voice agent without spending weeks tuning separate speech recognition, reasoning, turn-taking and speech synthesis components. Eleven Realtime offers a clear product choice: Mini for most everyday conversations and Max for demanding instructions and tool-heavy work.
The benchmark results support that split. Mini's 95.8% Full-Duplex-Bench result is strong for conversational dynamics, while Max's 98.8% Big Bench Audio and 87.1% tool-use completion make it the more compelling choice for agents that must reason and act. Both report sub-second average time to first audio in the published tests.
The main reasons to pause are implementation constraints and cost. Max is not available in India or Singapore data-residency environments, and the Realtime API is less flexible than a fully custom stack when you need a specific LLM or speech component. Pricing also combines per-minute platform usage with token billing, so calculate total cost from a representative pilot.
For a product team that wants to add spoken interaction quickly, Realtime is a strong option. For a team with specialized infrastructure, compliance needs or carefully tuned component models, a custom architecture may still be the better fit.
18. Final Verdict
Eleven Realtime is a practical step toward simpler voice-agent development. Its main value is not a single benchmark record; it is packaging speech understanding, response generation, turn-taking and voice output into a managed system that developers can configure without wiring every component themselves.
The two launch models serve different needs. Mini is a sensible default for assistants, tutors and interactive characters because it prioritizes cost and natural conversation. Max is the stronger choice for customer support, lead qualification and multi-step workflows, with substantially better published scores on spoken reasoning and tool-use tasks.
The benchmark table gives a useful summary: Max scores 98.8% on Big Bench Audio and completes 87.1% of tool-use tasks, while Mini scores 75.9% and 60.3%. Mini narrowly leads on Full-Duplex-Bench, at 95.8% versus 94.1%. First-audio latency is 0.57 seconds for Max and 0.62 seconds for Mini in the reported tests.
My practical recommendation is to start with Mini, test it on real conversations, and upgrade to Max only when the task requires extra reasoning and tool reliability. Before production, verify pricing, workspace availability, API compatibility and data-residency requirements.
Bottom line: Eleven Realtime is worth testing if you are building voice AI agents and want less integration work. Mini is the best starting point for cost-sensitive conversation; Max is the better option for complex, tool-using agents where completing the task matters more than minimizing model cost.
Frequently Asked Questions
What is Eleven Realtime?
Eleven Realtime is ElevenLabs' family of real-time speech-to-speech models designed to simplify building low-latency voice agents.
What is the difference between Mini and Max?
Mini prioritizes lower cost and natural everyday conversation. Max is designed for complex instructions, reasoning and multi-step tool use.
How accurate is Eleven Realtime
On Big Bench Audio, Max scored 98.8% and Mini scored 75.9%. Results vary by task, language and deployment conditions.
How fast is Eleven Realtime?
ElevenLabs reports average time to first audio of 0.57 seconds for Max and 0.62 seconds for Mini in its benchmark evaluation.
Does Eleven Realtime support tool calling?
Yes. Published tool-use completion scores are 87.1% for Max and 60.3% for Mini.
How many languages does it support?
ElevenLabs says both models support more than 90 languages.
Can I use it with the OpenAI Realtime SDK?
The API follows the OpenAI Realtime API wire protocol. Developers can point compatible clients at the ElevenLabs endpoint, then test session, audio and authentication behavior.
How much does it cost?
Realtime agents use the standard ElevenAgents per-minute rate plus model input, cached-input and output token usage. Max costs more per token than Mini.
Can I use Eleven Realtime in India?
Mini is listed as available in India data-residency environments. Max is currently unavailable in India and Singapore data-residency environments.
Is Eleven Realtime a fused speech model?
No. ElevenLabs documents Mini and Max as using a cascaded architecture and does not disclose the underlying component models.
Should I choose Mini or Max?
Start with Mini for common conversations. Choose Max when complex reasoning, long instructions or multi-step tool calls are essential.
Is Eleven Realtime better than a custom voice stack?
It is simpler to configure, while a custom stack offers more control over individual ASR, LLM, TTS and orchestration components.
Recommended Blogs
Mai Transcribe 2 Review: Features, Accuracy & Use Cases (2026)
How to Secure AI Coding Agents: Permissions, Sandboxing, MCP & Secrets
Replit's AI Model Routing Is Here: How Intelligent Routing Cuts Costs (2026)
Resources & Community
Join our community of 70,000+ AI enthusiasts and learn to build powerful AI applications. Build Fast with AI helps creators, developers and teams understand and implement practical AI.
Agentic AI Launchpad 2026
A structured 6-week cohort program for learning to build and deploy real-world agentic AI systems, with live sessions, expert mentorship, project reviews and a builder community.
Ready to go from learning to building? Join the next cohort: Agentic AI Launchpad 2026
Free AI Resources
Access free tools, workshops and micro-learning to keep building.
