Build a Voice Agent That Books Appointments in Under 1 Hour Using Claude Code and ElevenLabs

A Voice Agent That Books Real Appointments — Built in Under an Hour

Configuring a voice agent by hand is tedious in a specific way. You open the ElevenLabs dashboard, write a system prompt, pick a voice, figure out the Cal.com API, wire up two separate tool definitions, copy an embed snippet, and then discover the agent is querying availability in UTC when your clients are in Central time. Each step is individually manageable. Together they eat an afternoon.

This post walks through an end-to-end build where Claude Code does most of that work for you: it reads the ElevenLabs docs, creates the agent, adds Cal.com booking tools, embeds the widget, and — this is the part worth paying attention to — debugs a timezone bug by reading the raw conversation transcript. The whole thing, including iteration, runs under one hour.

The agent we’re building is a sales assistant that lives on a website, answers questions about your business, and books discovery calls directly into your calendar. No middleware. No Zapier. ElevenLabs calls Cal.com directly.

---

What you need before you start

You’ll need accounts on three services and one local tool:

• ElevenLabs — any paid plan gives you API access and the voice agent feature. If you have a professional voice clone trained there, you can use it. If not, pick any voice from their library.
• Cal.com — free tier works. You need at least one event type created (e.g., a 30-minute discovery call) and you need to know its event type ID.
• Claude Code — requires a paid Claude subscription. Install it as a VS Code extension (search “Claude Code” in the extensions panel).
• A project folder — this can be an existing Next.js site or a blank folder. Claude Code will add files to whatever’s there.

You’ll also want a .env file ready to receive two keys: your Cal.com API key and your ElevenLabs API key. Claude Code will create the file with placeholder values — you just paste in the real ones.

One thing to check before you start: look at your Cal.com event type’s minimum notice setting. If it’s set to 2 hours (the default), the first bookable slot will always be 2 hours from now. That’s not a bug — it’s a setting. But it will confuse you during testing if you don’t know it’s there.

---

Building the agent, step by step

Step 1: Open Claude Code in plan mode and describe what you want

Open VS Code, open your project folder, and launch Claude Code from the top-right panel. Switch it to plan mode before you type anything. Plan mode means Claude will ask clarifying questions and draft an architecture before writing any code.

Then describe your goal in plain language. Something like:

I want to embed a voice agent widget on my website. The agent should answer questions about my business and book discovery calls. I want to use ElevenLabs for the voice layer and Cal.com for booking. The agent should collect the visitor’s name, email, company, and the problem they’re trying to solve. It should check availability and book the call directly — no middleware.

Claude Code will come back with questions: Do you have an ElevenLabs account? Do you have a Cal.com event type ready? How should the widget appear — floating bubble or inline? What voice persona do you want?

Answer each one. The more specific you are here, the less iteration you’ll need later. “Warm, professional B2B sales tone” is better than “professional.” “Floating bubble, bottom right” is better than “on the page somewhere.”

Now you have: a shared understanding of the goal, and Claude Code has enough context to draft a real plan.

Step 2: Review the plan and get your API keys

Claude Code will return a structured plan. It should include:

1. Cal.com prep — API key, event type ID
2. ElevenLabs agent creation — system prompt, voice selection, first message
3. Tool configuration — check_availability and book_appointment
4. Widget embed — the HTML snippet added to your site

Read through it. The system prompt Claude drafts at this stage is usually good enough to start with. You’ll refine it through testing.

Now go get your keys:

Cal.com: Settings → API Keys → Create new key. Name it something like “voice-agent-demo.” Copy the key.

ElevenLabs: Settings → Developers → API Keys → Create new key. Make sure you grant it permissions (or disable restrictions for testing). Copy the key.

Open the .env file Claude Code created. It will look like this:

CAL_API_KEY=your_cal_api_key_here
ELEVENLABS_API_KEY=your_elevenlabs_api_key_here

Paste in your real values. Save. Do not commit this file to GitHub — add .env to your .gitignore if it isn’t already there.

Tell Claude Code: “I just dropped in both API keys. Keep going.”

Now you have: authenticated access to both services, and Claude Code can proceed without you.

Step 3: Watch Claude Code build the agent

This is where you mostly wait. Claude Code will:

• Call the ElevenLabs API to create a new agent
• Set the persona (system prompt), voice, and first message
• Add two tools to the agent: check_availability and book_appointment, both pointed at Cal.com
• Retrieve your Cal.com event type ID and wire it into the tool definitions
• Add the ElevenLabs widget embed snippet to your site

When it finishes, go into your ElevenLabs dashboard and click on Agents. You should see a new agent there. Click into it, then click Tools — you should see both Cal.com tools configured with the correct endpoint and parameters.

If you’re curious about the system prompt, click Agent → Persona. Claude Code will have written something reasonable. You’ll tune it later.

Now you have: a live ElevenLabs agent with Cal.com booking tools, and a widget embedded in your local site.

Step 4: Test the agent and take notes on what’s wrong

Run your site locally. Hard-refresh the page. You should see the widget — a floating bubble in the corner. Click Start call.

Talk to it. Try to book a call. Pay attention to:

• Does it give you a first message, or does it wait for you to speak first?
• Does the voice sound right?
• When you ask for availability, does it return reasonable time slots?
• Does it spell back your name and email before booking?

Write down everything that’s off. Don’t try to fix things mid-call. The goal of this first test is observation.

Common issues at this stage: the first message not triggering (an ElevenLabs configuration detail), the voice being wrong, the agent not confirming spelling before booking.

Now you have: a working agent and a concrete list of things to improve.

Step 5: Iterate with natural language

Go back to Claude Code. Describe what you observed:

The first message isn’t triggering — I have to start the conversation. The voice sounds too enthusiastic. The agent needs to spell back the name and email character by character before booking, because it was getting emails wrong. Make it more concise overall.

Claude Code will research the first-message issue in the ElevenLabs docs, make the changes via API, and update the system prompt. This is the loop: test, observe, describe, fix.

If you’re doing a lot of iteration, use a session handoff before starting a new debugging thread. This clears the context window and injects a summary of where you are — it prevents the kind of context drift that makes Claude Code start contradicting its earlier decisions. You can build this as a simple Claude Code skill; if you want a pre-built version, the Claude Code skills vs plugins breakdown explains how to package and reuse these patterns.

Now you have: a refined agent that handles the basics correctly.

Step 6: Debug the timezone bug (this one’s interesting)

After a few rounds of iteration, you may hit a situation where the agent reports fewer available slots than you expect. In the demo this post is based on, the agent said only 6:30 p.m. was available when the calendar showed openings from 4:00 p.m. to 9:00 p.m.

There are three possible causes:

1. Cal.com is returning fewer slots than expected
2. The agent is querying the wrong time window
3. The agent is misreading the tool output

Instead of manually inspecting API logs, tell Claude Code what you experienced and point it at the conversation transcript. In ElevenLabs, every conversation is logged — you can read the full transcript including tool calls.

Claude Code found the bug at turn 16 of the transcript: the check_availability tool was constructing its time window in UTC instead of Central time. The search window was offset by 5 hours, which is why early afternoon slots were invisible. Claude Code fixed the tool parameter to pass the correct timezone, and the next test showed full availability.

This is one of the more useful things about keeping your agent configuration in code rather than clicking through a dashboard: Claude Code can read the transcript, identify the exact tool call that failed, and fix the parameter — without you having to understand the Cal.com API yourself.

Now you have: a correctly functioning booking tool that queries availability in the right timezone.

---

What can go wrong (and how to handle it)

The widget doesn’t appear after embed. Hard-refresh the page. If it still doesn’t show, check that the ElevenLabs agent ID in your code matches the agent you created. Claude Code sometimes creates a new agent and forgets to update the widget snippet.

The first message never plays. This is an ElevenLabs configuration issue. The first message field in the dashboard may be set, but the widget needs a specific initialization parameter to trigger it automatically. Tell Claude Code the symptom — it will look up the right fix in the ElevenLabs docs.

Cal.com returns no slots. Check two things: (1) your availability hours in Cal.com — if you only have 9am–5pm configured, evening slots won’t appear; (2) the minimum notice setting — if it’s 2 hours and you’re testing at 4pm, the first slot will be 6pm.

The agent books with the wrong email. This is a prompting issue. The agent needs explicit instructions to spell back the email character by character and confirm before booking. Add this to the system prompt.

Latency feels high during local testing. This is normal. Local development adds overhead. Once the site is deployed to a real domain, latency drops noticeably. Premium voices with larger LLMs will always have more latency than smaller models — there’s a real tradeoff there.

Someone steals your widget. The ElevenLabs widget embed is a plain HTML snippet. Anyone who inspects your page source can copy it and run your agent on their own site, burning your credits. Lock it down: go to Security in the ElevenLabs dashboard and add your domain to the hostname allow list. Also check the Widget tab — there’s a separate allow-domains field there. Do both.

---

Where to take this further

Once the agent is working locally, deploying it is straightforward. Push your project to a GitHub repo, import it into Vercel, set the Next.js preset, and add your environment variables (Cal.com API key and ElevenLabs API key) in the Vercel dashboard. The .env file stays local — never committed.

A few things worth adding before you go public:

Rate limiting. If your widget is on a public page, set a per-IP rate limit. An uncapped public widget can be hit thousands of times by bots. ElevenLabs lets you set a max call duration per session — use it.

Knowledge base. Right now the agent only knows what’s in its system prompt. If you want it to answer detailed questions about your services, upload a document to the ElevenLabs knowledge base. The agent will query it during conversations.

Phone number. The same agent you just built can be connected to a Twilio phone number. Someone calls your business number, the agent picks up, books the call. Same engine, different entry point.

Multi-agent orchestration. If you’re building something more complex — say, a booking agent that also qualifies leads and routes them to different calendars based on company size — you’ll start thinking about how agents hand off to each other. Platforms like MindStudio handle this kind of orchestration with 200+ models and 1,000+ integrations, letting you chain agents and tools visually instead of stitching APIs by hand.

If the project grows into a full application — user authentication, a dashboard for reviewing bookings, CRM sync — the abstraction layer above TypeScript becomes relevant. Remy takes a different approach to that problem: you write an annotated markdown spec describing your application, and it compiles a complete full-stack app from it — TypeScript backend, SQLite database, auth, deployment. The spec is the source of truth; the code is derived output.

The voice agent itself is a good place to keep iterating. The prompting, the voice, the qualification questions — these all improve with real conversations. The advantage of having everything in code is that Claude Code can read your transcripts, find what’s going wrong, and fix it without you having to become an expert in ElevenLabs internals.

If you want to build more sophisticated Claude Code workflows, the Claude Code AutoResearch approach for self-improving skills is worth reading — it’s a pattern for automatically improving prompt quality over time using real outputs as feedback. And if you’re thinking about how to give your agent persistent memory across sessions, the self-evolving Claude Code memory system with Obsidian post covers a practical approach using hooks to capture and reuse session knowledge.

The build described here — agent creation, Cal.com tools, widget embed, timezone debug — took about 45 minutes of active work in the demo. Most of that time was testing and iteration, not configuration. That ratio is the point.