Most AI assistants are text boxes. You type, it types back. That works, but it's not how people naturally communicate. Voice changes the dynamic completely.
I've been running OpenClaw, an open-source self-hosted AI gateway, and connected it to ElevenLabs for text-to-speech. Here's what I've learned about voice-enabled AI agents: what works, what's still hard, and where the real opportunities are.
How OpenClaw + ElevenLabs Works
OpenClaw is an always-on AI gateway you host yourself. It connects to your chat apps (WhatsApp, Telegram, Discord, Signal) and gives your AI agent memory, tools, and access to your files. Think of it as the brain layer.
ElevenLabs is the voice layer. When OpenClaw generates a response, ElevenLabs converts it to natural-sounding speech in real-time. The integration supports streaming playback, so audio starts playing before the full response is generated. That keeps latency low.
The setup supports two voice modes:
TTS on demand. The agent decides when voice makes sense. Telling a story? Voice. Answering a quick question? Text. The agent can switch voices mid-conversation, control speed and style, and even use different voices for different characters. This works across all platforms — your phone gets an audio message in WhatsApp, Discord, or Telegram.
Talk Mode. A continuous voice conversation loop on macOS, iOS, and Android. You speak, OpenClaw transcribes, processes, and speaks back. It supports interrupt detection — start talking while the agent is speaking and it stops to listen. Latency on a decent connection sits under two seconds.
There's also Voice Wake — configurable wake words (like "hey openclaw" or just "computer") that activate Talk Mode hands-free on macOS and iOS. Wake words are stored on your gateway and synced across all your devices.
What Voice Actually Enables
Voice isn't just a novelty. It unlocks use cases that text can't match:
Storytelling and content. An AI agent reading a bedtime story, narrating a movie summary, or delivering a podcast script hits different than a wall of text. You can assign character voices, control pacing, add dramatic pauses. ElevenLabs' voice quality is close enough to human that it doesn't feel robotic.
Hands-free workflows. Cooking, driving, walking, working with your hands. These are all situations where typing isn't an option but talking is. Morning briefings, calendar checks, quick status updates — all natural voice interactions.
Accessibility. Not everyone prefers reading. Voice output makes AI agents usable for people with visual impairments, reading difficulties, or anyone who simply processes information better by listening.
Personality. A text response is flat. A voiced response has tone, rhythm, warmth. It makes the AI agent feel more like a colleague and less like a search engine. You pick the voice that fits — professional, casual, warm, or something entirely custom with a voice clone.
The Challenges (Being Honest)
Voice-first AI isn't seamless yet. Here's what's hard:
Speech-to-text is the weak link. TTS quality from ElevenLabs is excellent. STT (transcription) still struggles in noisy environments — cars, cafés, wind. This is an industry-wide problem, not specific to any one tool, but it matters when you're building a voice-first workflow. Background noise kills the magic fast.
Norwegian language support. ElevenLabs handles English beautifully. Norwegian? It works, but accents, intonation, and lesser-used words can sound off. If you're building for a Norwegian audience, expect to test and tweak. The gap is closing quickly though — `eleven_v3` is noticeably better than v2 was.
Cost scales with usage. ElevenLabs charges by character. Casual use is cheap ($5/month covers a lot). But if you're having long voice conversations daily or generating content at scale, costs add up. You need to be intentional about when voice adds value vs. when text is fine.
Setup isn't plug-and-play. Self-hosting means you're managing a VPS, configuring API keys, and pairing devices. The payoff is full control and privacy, but the barrier to entry is higher than downloading an app. Technical users will be fine; mainstream adoption needs more polish.
Latency is good, not instant. Sub-two-second response times feel responsive for most interactions. But for rapid back-and-forth conversation, there's still a noticeable gap compared to talking to a human. Streaming playback helps, but it's not zero-latency.
The Opportunities
This is where it gets interesting:
Custom voice agents for businesses. Imagine a customer support agent that sounds consistent, speaks your brand's language, and has access to your knowledge base. OpenClaw provides the AI brain and tool access; ElevenLabs provides the voice. No vendor lock-in, fully self-hosted.
Multilingual agents. ElevenLabs supports dozens of languages with the same voice. One agent, one voice, multiple languages. For companies operating across borders, this is powerful. The agent can detect the user's language and respond in kind.
Voice as a content creation tool. Blog posts as audio articles. Documentation as spoken guides. Meeting summaries as voice memos. The agent generates the content and voices it automatically. No recording studio needed.
Proactive voice notifications. Instead of push notifications you ignore, imagine your agent calling out important updates in a natural voice. "Your meeting with Kyvco starts in 15 minutes" or "That deploy just failed" — delivered as speech to your earbuds.
Education and training. AI tutors that explain concepts by speaking, adjust their pace based on the learner, and switch between languages. Voice makes learning feel more personal than reading a chatbot response.
Where This Is Heading
The pieces are converging fast. Voice synthesis is nearly indistinguishable from human speech. AI models are getting better at knowing when to speak vs. type. Self-hosted gateways like OpenClaw give you the control layer. Wake words and always-on listening are maturing on mobile.
The missing piece has been the glue — something that connects the AI brain, the voice layer, the chat platforms, and your personal data into one coherent system. That's the gap tools like OpenClaw fill.
Within a year, I expect voice-first AI agents to be the default for technical users. Within two, the setup friction will drop enough for mainstream adoption. The question isn't whether this will happen — it's who builds the best experience first.
Getting Started
If you want to explore this yourself:
1. OpenClaw: `npm install -g openclaw@latest` → `openclaw onboard --install-daemon` 2. ElevenLabs: Grab an API key from elevenlabs.io (free tier available) 3. Configure: Add your voice ID and API key to `openclaw.json` under `talk` 4. Pair your phone: OpenClaw companion apps for iOS and Android
Full docs at docs.openclaw.ai. Community at discord.gg/clawd.
The best part: you own everything. Your data, your voice, your infrastructure. No subscription to cancel, no terms of service to worry about, no company deciding to deprecate your workflow.