How to Build a Voice Agent with 11 Labs and Cal.com Booking Using Claude Code: 45-Minute Walkthrough

A Working Voice Booking Agent in 45 Minutes, Without Reading a Single API Doc

Configuring a voice agent by hand is tedious in a specific way. You open the 11 Labs dashboard, write a system prompt, pick a voice, figure out the tool schema, paste in an endpoint URL, realize you got a parameter wrong, go back to the Cal.com docs, fix it, forget to save, start over. It takes an afternoon and you still aren’t sure if the timezone handling is correct.

There’s a faster path. The 11 Labs voice agent + Cal.com booking integration you’ll build in this walkthrough took approximately 45 minutes using only natural language prompts to Claude Code — no manual dashboard configuration, no reading API documentation, no hand-writing JSON schemas. Claude Code reads the docs itself, asks you clarifying questions, writes the configuration, and tells you when something breaks.

This tutorial walks through exactly how that build happened: what prompts triggered what actions, where things broke, how the debugging worked, and what the finished agent actually does.

---

What you get at the end

The finished agent is a voice widget embedded on a website. A visitor clicks “Start call,” talks to an AI sales agent, and the agent can:

• Answer questions about the business from a knowledge base
• Collect the visitor’s name, email, company, and problem statement
• Check real-time availability on a Cal.com calendar
• Book a 30-minute discovery call and send a confirmation email

The agent runs on 11 Labs’ voice infrastructure with a professional voice clone. The Cal.com integration uses two tool calls: check_availability and book_appointment. The whole thing embeds on any webpage with a single HTML snippet.

If you’ve ever tried to build something like this manually, you know the gap between “I want a voice booking agent” and “I have a voice booking agent” is usually measured in days, not hours.

---

What you need before starting

Accounts:

• Claude Code (requires a paid Anthropic subscription)
• 11 Labs account (free tier works to start; you’ll need API access)
• Cal.com account with at least one event type configured and synced to your calendar

Tools:

• VS Code with the Claude Code extension installed
• A project folder — even a simple landing page HTML file is enough

Nice to have:

• A voice-to-text dictation tool for faster prompting. The builder in this walkthrough uses Glido, which is faster and more private than Whisper-based alternatives. Dictating prompts instead of typing them meaningfully speeds up the iteration loop.

You do NOT need to read the 11 Labs API docs or the Cal.com API docs. That’s the point.

---

The build, step by step

Step 1: Open Claude Code in plan mode and describe the goal

Before Claude Code writes a single line, switch it to plan mode. This tells Claude to ask clarifying questions instead of immediately executing. It’s a small toggle with a big effect — you want alignment before action.

Then describe your goal in plain language. Don’t worry about being technically precise. Something like this works:

I have a website for an AI consultancy. I want to embed a voice agent widget using 11 Labs. The agent should answer questions about the business, act as a sales agent, and ultimately push people to book a discovery call. I want to connect it to Cal.com so it can check availability and book calls. The visitor would give their name and email and the agent would book it directly.

Claude Code will respond with clarifying questions. In this build, it asked five:

1. What’s your current 11 Labs setup? (account but no agent yet)
2. What’s your Cal.com setup? (account with event type ready)
3. How should the booking happen — direct from 11 Labs to Cal.com, or via a middleware workflow?
4. How should the widget appear? (floating bubble)
5. What voice persona do you want? (warm, professional B2B sales tone)
6. What data fields beyond name and email should the agent collect? (company name, problem statement, team size)

Answer these. Claude Code then returns a full architecture plan before touching anything.

Now you have: A shared plan — agent name (Neural Diagnostics), voice persona, two Cal.com tools, widget embed strategy — all confirmed before any code runs.

---

Step 2: Create the .env file and drop in your API keys

Claude Code will tell you it needs two API keys and will create a .env file with placeholder values. The .env is automatically excluded from git commits.

Go get the keys:

Cal.com: Settings → API Keys → Create new key → copy it

11 Labs: Settings → Developers → API Keys → Create new key → grant permissions (or disable restrictions for a demo key) → copy it

Paste both into the .env file Claude Code created. Save. Tell Claude Code: “I just dropped in those API keys. Keep going.”

Now you have: Authenticated access to both APIs, stored locally, not committed to version control.

---

Step 3: Let Claude Code build the agent

With the plan confirmed and keys in place, Claude Code works through its to-do list autonomously. It:

• Queries Cal.com to find your event type and confirms it’s using the 30-minute slot
• Creates an 11 Labs agent via the API with a full system prompt, first message, and voice selection
• Configures two tools on the agent: check_availability and book_appointment, with correct Cal.com endpoint schemas
• Wires the widget embed snippet into your website’s HTML

You don’t touch the 11 Labs dashboard during this step. Claude Code reads the 11 Labs documentation itself and makes the API calls directly.

When it’s done, open your 11 Labs dashboard and go to Agents. You’ll see “Neural Diagnostics” (or whatever name you chose) with a full system prompt already written and two tools already configured. This is the part that normally takes an afternoon.

Now you have: A live 11 Labs voice agent, configured entirely via API, with Cal.com booking tools attached.

---

Step 4: Test the widget and iterate on the voice and prompting

Open your site (localhost is fine), hard refresh, and click the widget. Talk to it.

The first test will probably reveal a few things to fix. In this build:

• The default voice (Adam) felt too enthusiastic and AI-sounding
• The first message wasn’t triggering — the agent waited for the user to speak first
• The agent was reading back emails using NATO phonetic alphabet, which slowed down the confirmation flow significantly

None of these require you to touch the dashboard. You tell Claude Code what you noticed:

Change the voice — this one sounds too AI. Fix the first message not triggering. Remove the NATO phonetic spelling from the system prompt, it’s making the email confirmation too slow.

Claude Code makes the changes via the 11 Labs API. It also reduces the temperature slightly to get more consistent outputs after you ask for more concise responses — a small tuning detail that makes a real difference in how the agent sounds.

Now you have: An agent with a better voice, a working first message, and a tighter system prompt.

---

Step 5: Debug the availability tool

This is where the build gets interesting.

After the voice and prompting fixes, test the booking flow. Ask the agent what slots are available. In this build, the agent said only one slot was available (6:30 PM) when the calendar showed open time from 4:00 PM to 9:00 PM.

Something was wrong with the check_availability tool call. There were three possible causes:

1. Cal.com was only returning one slot
2. The agent was querying a too-narrow time window
3. The agent was misreading Cal.com’s response

Rather than digging through the 11 Labs dashboard manually, describe the symptom to Claude Code and ask it to investigate. Claude Code reads the conversation transcript from the 11 Labs dashboard — the actual tool call logs — and finds the bug.

The agent had constructed the availability query in UTC, not the user’s local timezone (Central). So when the user asked for “tonight,” the agent was querying the wrong time window entirely. Claude Code found this by reading turn 16 of the conversation transcript, without being told to check the tool parameters specifically.

This is the kind of bug that’s genuinely hard to find manually. You’d have to know to look at the raw tool call, know what UTC offset to check, and know which parameter was wrong. Claude Code found it by reading the transcript the same way you would — just faster.

Fix the timezone parameter, test again. This time the agent returns the correct slots: 6:30 PM, 7:00 PM, 7:30 PM, 8:00 PM, 8:30 PM Central.

One thing to know: Cal.com has a minimum notice setting (default: 2 hours). If it’s currently 4:30 PM, the first available slot will be 6:30 PM — not because the tool is broken, but because Cal.com is enforcing the 2-hour buffer. This is a hidden constraint that affects which slots appear. Check your Cal.com event settings under Limits if you’re seeing fewer slots than expected.

Now you have: A working availability check that queries in the correct timezone and respects Cal.com’s booking constraints.

---

Step 6: Complete a full booking and verify

Run the full booking flow end to end. The agent should:

1. Greet the visitor
2. Collect name (and spell it back for confirmation)
3. Collect email character by character
4. Ask about company and problem
5. Check availability and offer slots
6. Book the selected slot
7. Confirm the booking with the email address

In the final test: the visitor asked for 7:00 PM Central, the agent booked it, and a confirmation email arrived at the provided address within seconds.

Now you have: A fully working voice booking agent — persona, voice, knowledge base, and two Cal.com tools — embedded on a website and confirmed end-to-end.

---

The failure modes worth knowing about

The first message doesn’t trigger. This happened in this build and required Claude Code to look up the 11 Labs documentation to fix. If your widget loads but the agent waits for the user to speak first, the first message configuration isn’t being applied correctly via the API. Tell Claude Code specifically: “The first message isn’t triggering. Look up the 11 Labs docs and fix it.”

Wrong timezone in tool calls. The UTC bug described above is easy to miss. If your agent is returning fewer slots than expected, ask Claude Code to read the conversation transcript and check what parameters the tool call actually sent.

Cal.com minimum notice filtering slots. If you’re testing in the afternoon and only see evening slots, check your Cal.com event’s Limits settings. The minimum notice period (often 2 hours by default) silently filters out near-term availability.

Voice sounds too AI. The default voices in 11 Labs vary a lot. If the first voice doesn’t feel right, just say so. Claude Code can swap it via the API without you touching the dashboard. If you have a professional voice clone (trained on ~4 hours of audio), you can specify that instead.

NATO phonetic alphabet in email confirmation. If you ask the agent to spell back an email and it reads “November-Alpha-Tango-Echo,” that’s the system prompt being overly cautious. Remove the NATO phonetic instruction and let the agent read characters directly. Much faster.

Context getting long during iteration. After several rounds of debugging, the Claude Code context window fills up. A session handoff — asking Claude Code to summarize where things stand, then starting a fresh session with that summary — keeps the context clean and prevents degraded outputs. This is especially useful if you’re spending more than an hour iterating.

---

Where to take this further

The agent you’ve built is a starting point. A few directions worth considering:

Security before going live. The 11 Labs widget is a snippet of HTML. Anyone who inspects your page source can copy it and run your agent on their site, at your cost. Lock it down: add your domain to the 11 Labs allowlist (Settings → Security → Allowed Domains), set a per-call duration cap, and configure rate limiting if the page is public. Claude Code can help implement all of this — just describe what you want.

Phone deployment. The same agent can answer a phone number. Connect a Twilio number to your 11 Labs agent and callers reach the same booking flow without visiting your website. Same system prompt, same tools, different entry point. If you’re interested in how Claude Code handles multi-channel agent deployment, the patterns in Beyond One-Shot Prompts: 5 Claude Code Workflow Patterns apply directly here.

Knowledge base depth. Right now the agent answers questions from whatever you put in the 11 Labs knowledge base. For a real consultancy, you’d want to feed it actual service descriptions, case studies, and FAQs — not just a system prompt. Claude Code can help structure and upload that content.

Persistent memory across sessions. Voice agents don’t remember previous callers by default. If you want the agent to recognize returning visitors or remember past conversations, you need a memory layer. The approach in How to Build a Self-Evolving Claude Code Memory System With Obsidian and Claude Code Hooks shows one way to build that kind of persistent context.

Connecting to your CRM. After a call is booked, you probably want that lead in HubSpot or Salesforce. Platforms like MindStudio handle this kind of orchestration — 200+ models, 1,000+ integrations, and a visual builder for chaining agents and workflows — which is useful when the voice agent is one piece of a larger pipeline rather than a standalone tool.

Building more complex booking logic. If your scheduling needs get more complicated — different meeting types, routing logic, multi-step qualification — you’ll eventually want to move beyond a single Cal.com event type. That’s when the spec-driven approach of tools like Remy becomes relevant: you write your booking logic as annotated markdown, and it compiles into a full TypeScript backend with database and auth, rather than stitching together more API calls.

Iterating on the prompt. The system prompt Claude Code wrote is a starting point. The agent in this build was still asking too many questions in sequence, which made the demo feel slow. Prompt iteration — making the agent more direct, adjusting when it asks for information versus when it just proceeds — is ongoing work. The Claude Code for Content Marketing: How to Build a Skill-Based Content Machine post has a useful framing for building reusable skills that carry prompt improvements forward across sessions.

The 45-minute build gets you to “technically working.” Getting to “feels natural and converts visitors” takes more iteration — but all of that iteration happens the same way: describe what you noticed, let Claude Code fix it, test again. You never have to open the API documentation.

That’s the actual shift here. Not that the agent is built faster, but that the feedback loop is tighter. You spend your time talking to the agent and noticing what’s wrong, not reading docs and configuring endpoints. For anyone who’s built voice agents the manual way, that difference is significant.