11 Labs Voice Agent via API: 4 Components Claude Code Configures Without You Touching the Dashboard
Persona, voice, knowledge base, tools — all four 11 Labs agent components configured entirely through Claude Code. Here's the full API-first workflow.
4 Components, Zero Dashboard Clicks: How Claude Code Configures an 11 Labs Voice Agent via API
Most developers building a voice agent in 11 Labs spend their first hour clicking through the dashboard — setting the system prompt here, picking a voice there, wiring up a tool in a third tab, hoping they didn’t forget to save. There’s a faster path. Four voice agent components — persona, voice, knowledge base, and tools — can be configured entirely through the 11 Labs API, with Claude Code handling every call. You never have to open the dashboard to build a working agent.
That’s the workflow Nate Herk demonstrated recently: a fully functional 11 Labs voice agent with Cal.com booking integration, built and debugged in approximately 45 minutes using only natural language prompts. No manual API documentation reading. No dashboard configuration. Claude Code read the 11 Labs docs, wrote the API calls, created the agent, and wired up two Cal.com tools — all from a single project directory.
Here’s how each of the four components works in an API-first build, and what Claude Code actually does for each one.
Component 1: Persona — The System Prompt as Code
The persona is the agent’s system prompt. In the dashboard, you’d type it into a text box. In an API-first build, Claude Code writes it as a string and pushes it to the 11 Labs agent creation endpoint.
Remy doesn't write the code. It manages the agents who do.
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
The practical difference isn’t just convenience. When the persona lives in code — or at minimum in a file Claude Code can read and edit — you can iterate on it the same way you’d iterate on any other configuration. Herk’s build went through several persona revisions during the 45-minute session: the agent’s tone shifted from generic to “warm, professional B2B sales,” the temperature was reduced to produce more consistent outputs after a conciseness request, and a specific instruction was removed mid-session.
That last change is worth dwelling on. The original system prompt included NATO phonetic spelling — the idea being that the agent would spell out email addresses letter by letter using Alpha, Bravo, Charlie, etc. In practice, it slowed down the email confirmation flow noticeably. Herk flagged it, Claude Code removed the instruction from the prompt and pushed the update. The whole cycle took under two minutes.
When the persona is a dashboard text box, that kind of iteration requires navigating to the right tab, editing, saving, and re-testing. When it’s a string in a file that an AI agent can modify on instruction, the feedback loop compresses significantly.
The agent in this build was named Neural Diagnostics, positioned as a B2B sales agent for an AI consultancy. The system prompt instructed it to capture company name, problem statement, team size, and role — in addition to the name and email required to complete a Cal.com booking.
Component 2: Voice — Selecting and Assigning Programmatically
Voice selection in 11 Labs is typically a browse-and-click operation. The API-first approach lets Claude Code assign a voice ID directly when creating or updating the agent.
Herk’s build used his professional voice clone — trained on approximately four hours of his own voice data — which lives in his 11 Labs account as a named asset with an ID. Claude Code can reference that ID in the agent configuration without Herk having to navigate to the voice library and copy anything manually.
The first test run used a default voice called Adam. Herk’s feedback was direct: “I don’t like this voice. It sounds too AI.” Claude Code updated the voice assignment to the professional clone. One instruction, one API call, done.
This matters more than it might seem. Voice selection is often treated as a cosmetic decision, but it’s actually a significant factor in whether a voice agent feels trustworthy to a prospect. A B2B sales agent that sounds like a generic text-to-speech engine will lose calls it should win. The ability to swap voices quickly — and to test the difference in a live widget without re-deploying anything — is a real workflow advantage.
For teams building voice agents at scale, the API-first approach also means voice assignment can be parameterized. Different agents for different use cases can pull from different voice IDs without anyone touching a dashboard.
Component 3: Knowledge Base — Grounding the Agent in Real Data
The knowledge base is where the agent gets its facts. Without it, a voice agent will hallucinate — confidently inventing details about your business, your pricing, your team. With it, the agent can answer questions accurately because it’s drawing from documents you’ve provided.
In the 11 Labs dashboard, you’d upload documents manually. Via the API, Claude Code can push knowledge base content programmatically — which means the knowledge base can be updated as part of a deployment script rather than a manual upload process.
Herk’s build loaded the agent with information about the Neural Diagnostics consultancy: what the business does, what problems it solves, who it serves. This grounded the agent’s responses when prospects asked questions before agreeing to book a call.
The security implication here is worth flagging. Herk mentioned it explicitly: knowledge-base grounding is a guard against the agent making things up, but it’s also a guard against the agent being manipulated into saying things that aren’t true. If a prospect tries to get the agent to confirm a price that doesn’t exist or a service that isn’t offered, a well-grounded agent will fall back to what’s in the knowledge base rather than improvising.
For anyone building agents that will talk to real customers — not just internal tools — this is the component that determines whether the agent is trustworthy or a liability.
Component 4: Tools — Two Cal.com Endpoints, Configured Without Reading the Docs
Tools are where the voice agent stops being a chatbot and starts being an agent. In this build, Claude Code configured two tools against the Cal.com API: check_availability and book_appointment.
check_availability queries the Cal.com API for open slots on a given event type. book_appointment takes a name, email, and time slot and creates the booking. Together, they let the agent handle the entire scheduling flow in a single conversation — no handoff to a human, no redirect to a booking page.
Claude Code wrote both tool definitions, including the parameter schemas that tell the 11 Labs agent how to call each one. It also created a .env file for the two required API keys — the Cal.com API key and the 11 Labs API key — and excluded that file from git commits automatically. The event type ID for the 30-minute meeting slot was passed in as a configuration value.
This is where the API-first approach shows its clearest advantage over dashboard configuration. In the dashboard, you’d add each tool manually, fill in the endpoint URL, define each parameter by hand, and hope the schema was correct. Claude Code reads the Cal.com documentation, writes the schema, and pushes it. If the schema is wrong, you describe the problem and it fixes it.
One schema problem did surface during testing — a UTC timezone bug where the agent was constructing availability queries in UTC rather than the user’s local timezone. The fix required updating the tool parameters, not the system prompt. Claude Code found it by reading the conversation transcript from the 11 Labs dashboard, identified the specific turn where the wrong timezone appeared, and corrected the parameter construction. That debugging loop — describe the symptom, let Claude Code find the cause — is a meaningful part of what makes this workflow fast. (The full story of how that bug was caught is worth reading separately, but the short version is: Claude Code found it without being told where to look.)
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
There’s also a hidden constraint worth knowing about: Cal.com’s minimum notice setting. In Herk’s account, minimum notice was set to 2 hours. When the agent queried availability at 4:30 PM, the first available slot came back as 6:30 PM — not because the calendar was full, but because Cal.com was enforcing the 2-hour buffer. The agent wasn’t broken; the constraint was just invisible until you knew where to look.
Deploying the Widget: One HTML Snippet, One Security Decision
Once the four components are configured, deployment is a single HTML snippet from the 11 Labs widget tab. You copy it, give it to Claude Code, and it drops it into your site. That’s the entire embed process.
The security layer requires one more decision: domain allowlisting. The widget snippet contains your agent ID. Anyone who inspects your page source can copy that snippet and embed your agent on their own site, running up your 11 Labs credits. The fix is to add your domain to the allowlist in the 11 Labs security settings — either through the dashboard or by telling Claude Code to handle it via the API.
Herk flagged several other security considerations worth implementing before a widget goes live: a per-call duration cap to limit how long any single conversation can run, rate limiting to prevent the widget from being hammered by automated requests, and knowledge-base grounding to prevent the agent from being manipulated into off-script responses. None of these are complicated to implement, but they’re easy to skip if you’re moving fast.
The same agent can be connected to a Twilio phone number with minimal additional configuration. The system prompt, voice, knowledge base, and tools all carry over — it’s the same agent, different entry point. For a B2B sales use case, that means the same Neural Diagnostics agent that handles website visitors can also answer inbound calls.
The Plan-Before-Execute Pattern
One detail from the build that’s easy to overlook: before Claude Code wrote a single line of code, it asked five clarifying questions. Current state of the 11 Labs account. Current state of the Cal.com setup. How the booking should be created. How the widget should appear. What voice persona was wanted.
This is the plan mode workflow — Claude Code in a reasoning-first state before it starts executing. The questions it asked shaped the entire architecture: direct 11 Labs-to-Cal.com integration rather than routing through an intermediary automation tool, a floating bubble widget rather than an inline embed, a warm professional tone rather than a formal or casual one.
The value of that upfront alignment is that the first build attempt was close enough to work that iteration was about refinement, not rebuilding. If you’re building agents that connect multiple APIs, understanding how workflows, agents, and tools fit together before you start executing saves significant time in the debugging phase.
For teams thinking about how this kind of spec-driven approach scales, Remy takes a related idea further: you write your application as annotated markdown — a spec where prose carries intent and annotations carry precision — and it compiles into a complete TypeScript backend, SQLite database, frontend, auth, and deployment. The source of truth is the spec; the code is derived output.
The Session Handoff Skill
Remy is new. The platform isn't.
Remy is the latest expression of years of platform work. Not a hastily wrapped LLM.
Herk used a session handoff skill between major iterations — a prompt that clears the context window and provides a summary of where the build stands. This kept Claude Code from accumulating stale context across multiple debugging cycles.
It’s a small operational detail, but it matters for longer builds. Context accumulation in an agentic session can cause Claude Code to anchor on earlier decisions that have since been revised. The session handoff forces a clean slate with only the relevant state carried forward.
For anyone running Claude Code sessions that extend past an hour — especially sessions that involve multiple rounds of debugging — keeping your Claude Code agent running cleanly across long sessions is worth thinking about before you start, not after you’ve hit a context problem mid-build.
What API-First Configuration Actually Changes
The dashboard isn’t going away. For one-off configuration, for exploring what’s possible, for checking that a tool was created correctly — the 11 Labs dashboard is useful. Herk used it to verify that the Neural Diagnostics agent appeared after Claude Code created it, and to check that both tools were listed under the agent’s configuration.
But the dashboard is a poor environment for iteration. Every change requires navigation, manual input, and saving. There’s no version history. There’s no way to describe a change in natural language and have it applied. There’s no way to batch-update multiple components at once.
The API-first approach inverts this. The dashboard becomes a read-only verification tool. All writes happen through Claude Code, which means all changes are describable in plain language, all changes can be batched, and all changes can be undone by describing what you want instead.
For teams building multiple voice agents — or building one agent and iterating it heavily — this is a meaningful operational difference. Platforms like MindStudio, which support 200+ AI models and 1,000+ pre-built integrations with a visual builder for chaining agents and workflows, approach the same problem from a different direction: abstracting the orchestration layer so you’re composing capabilities rather than writing API calls.
The four components — persona, voice, knowledge base, tools — are the same regardless of which path you take. What changes is how much of the configuration work you’re doing yourself versus delegating to a system that can read documentation, write schemas, and push updates on instruction.
Forty-five minutes to a working voice agent with live calendar booking is the benchmark Herk set. The dashboard path to the same result would take considerably longer — and would be harder to reproduce the next time you need to build one.
Before You Ship
A few things to verify before a voice agent goes live, based on what surfaced in this build:
The Cal.com minimum notice setting will silently constrain available slots. Check it before you assume the availability tool is broken. The domain allowlist in 11 Labs security settings is the difference between a widget that only runs on your site and one that anyone can copy and run on theirs. Temperature reduction in the system prompt produces more consistent outputs when you’ve asked the agent to be concise — the two instructions interact.
Remy doesn't build the plumbing. It inherits it.
Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something.
Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want.
And read your conversation transcripts. The UTC timezone bug in the Cal.com tool call was invisible until someone looked at the actual parameters being sent. Claude Code found it in the transcript. You can find it there too, if you know to look.
For teams thinking about browser automation and API integrations in Claude Code builds, the same transcript-reading approach applies to debugging web interactions — the artifact of what actually happened is almost always more informative than the error message.
The agent Herk built isn’t a prototype. It’s a deployable sales agent that books real meetings, sends real confirmation emails, and handles the full conversation flow from introduction to calendar invite. The four components are configured. The tools are working. The voice is right. That’s the bar.
