How to Build a Voice Agent That Books Meetings Using ElevenLabs and Calendly

What You’re Actually Building (And Why It Matters)

Scheduling back-and-forth is one of the most friction-filled parts of any sales or customer service process. A voice agent that can hold a real conversation, check availability, and book a meeting directly on your calendar removes that friction entirely — no forms, no links, no waiting.

This guide walks you through building a no-code voice agent that uses ElevenLabs for speech and Calendly for booking. The agent answers questions, qualifies intent, pulls available time slots, and confirms appointments — all in a natural spoken conversation.

Whether you’re building this for inbound sales, client onboarding, support triage, or lead capture, the core setup is the same. Let’s get into it.

---

What You’ll Need Before You Start

Before touching any configuration, make sure you have the following in place:

Accounts and access:

• A MindStudio account (free to start at mindstudio.ai)
• An ElevenLabs account with API access (their Creator tier works fine for most use cases)
• A Calendly account — the Standard or Teams plan gives you API access and event type control
• Basic familiarity with connecting tools through a visual builder (no coding required)

What you should decide upfront:

• Which Calendly event type the agent should book (e.g., a 30-minute discovery call)
• What information you want to collect from callers before booking (name, email, reason for calling)
• Whether the agent should handle multiple meeting types or just one

Having clear answers to these before you start saves significant time during configuration.

---

How the Voice Agent Works: The Full Picture

Before jumping into setup, it helps to understand what’s happening at each step of the conversation.

The Conversation Flow

A caller reaches your voice agent — either via a phone number, an embedded web widget, or a webhook trigger. The agent greets them, handles their question or request, and when the conversation reaches a booking moment, it:

1. Calls the Calendly API to pull available time slots
2. Presents the options verbally (“I have openings Tuesday at 2pm, Wednesday at 10am, or Thursday at 3pm — which works best for you?”)
3. Collects the caller’s name and email
4. Creates a Calendly invitee and confirms the booking
5. Sends a confirmation verbally and optionally by email

The caller never leaves the conversation. No links, no forms, no redirects.

The Technical Stack

• ElevenLabs handles text-to-speech (TTS) and optionally speech-to-text (STT), giving the agent a natural human-like voice
• Calendly API provides real-time availability and booking creation
• MindStudio acts as the workflow brain — it handles the conversation logic, connects to both services, manages state (like what slots were offered), and routes the flow based on caller responses
• A language model (GPT-4o, Claude, or similar) processes natural language, understands intent, and generates the responses the agent speaks

This combination means you’re not building a rigid phone tree — the agent can handle varied, messy, human conversation.

---

Setting Up Your Calendly Integration

Get Your Calendly API Credentials

Start in your Calendly account:

1. Go to Account Settings → Integrations → API & Webhooks
2. Generate a Personal Access Token
3. Copy the token — you’ll need it shortly

You’ll also need your Event Type URI. This is the specific meeting type the agent will book. To find it:

1. Use the Calendly API endpoint GET /event_types with your token
2. Identify the event type you want (e.g., your 30-minute discovery call)
3. Copy the uri value — it looks like https://api.calendly.com/event_types/XXXXXXXX

Keep both values somewhere handy.

Understand What the API Can Do

The Calendly API supports two critical actions for this use case:

• GET /event_type_available_times — Returns a list of available slots within a date range for a given event type
• POST /scheduled_events (via the booking link or directly) — Creates a new invitee and schedules the event

One important nuance: Calendly’s direct scheduling API (without using their hosted booking page) requires the Teams or Enterprise plan for full programmatic booking. On Standard, you can still pull availability and direct callers to a pre-filled link. Plan accordingly.

---

Configuring ElevenLabs for Voice

Choose and Clone a Voice

ElevenLabs gives you access to a large library of pre-made voices, and you can clone a custom voice from audio samples if you want something more brand-specific.

For a booking agent, you want a voice that sounds:

• Warm but professional
• Clear and easy to understand
• Natural in pacing (not robotic)

Their “Rachel” and “Drew” voices work well for most business contexts. Once you’ve selected one, grab the Voice ID from the ElevenLabs dashboard — it’s in the voice’s detail view.

Get Your ElevenLabs API Key

In your ElevenLabs account:

1. Go to Profile Settings
2. Copy your API Key

You’ll use this key to make TTS calls from MindStudio whenever the agent needs to speak a response.

Decide on Streaming vs. Standard Output

ElevenLabs supports both standard audio file generation and streaming audio. For a real-time voice agent, streaming is preferred — it reduces the gap between the agent finishing its response and the caller hearing it. MindStudio supports both modes depending on your deployment setup.

---

Building the Workflow in MindStudio

This is where the agent comes together. MindStudio’s visual builder lets you create the full conversation flow, connect to both ElevenLabs and Calendly, and handle branching logic — without writing code.

Create a New Agent

1. Log in to MindStudio and create a new AI Worker
2. Select a base language model — GPT-4o or Claude 3.5 Sonnet both perform well for conversational agents
3. Name your agent something descriptive (e.g., “Booking Agent — Discovery Calls”)

Set Your System Prompt

The system prompt defines the agent’s personality, scope, and behavior. Here’s a solid starting template:

You are a friendly scheduling assistant for [Company Name]. Your job is to help callers 
book a 30-minute discovery call.

When a caller expresses interest in booking, ask for:
1. Their preferred date and time (offer 3 options pulled from available slots)
2. Their first and last name
3. Their email address

Confirm the booking details before finalizing. Keep your responses short and conversational — 
you are speaking aloud, not writing an email. Never use bullet points or formatting in your 
responses. Speak naturally.

If the caller asks questions you can't answer, offer to connect them with a team member.

The last instruction about avoiding bullet points is important — formatted text sounds unnatural when spoken aloud.

Connect to Calendly

In MindStudio, add an HTTP Request block to your workflow:

• Method: GET
• URL: https://api.calendly.com/event_type_available_times
• Headers: Authorization: Bearer YOUR_TOKEN
• Query params:
  • event_type: your event type URI
  • start_time: dynamic date (e.g., today at 00:00 UTC)
  • end_time: dynamic date (e.g., 7 days from now)

This block runs when the agent detects booking intent. The response contains available time slots, which you pass back to the language model to present verbally.

For creating the booking, add a second HTTP Request block:

• Method: POST
• URL: https://api.calendly.com/scheduling_links or the direct invitee endpoint
• Body: Includes event type URI, invitee name, email, and selected time

Store collected values (name, email, chosen slot) as variables in MindStudio’s workflow state so they persist across turns in the conversation.

Add ElevenLabs for Voice Output

For every agent response, add a Text-to-Speech block that calls the ElevenLabs API:

• Endpoint: https://api.elevenlabs.io/v1/text-to-speech/{voice_id}
• Headers: xi-api-key: YOUR_API_KEY
• Body: { "text": "[agent response]", "model_id": "eleven_multilingual_v2" }

The audio output can be played back in a browser widget, streamed to a phone call, or returned via webhook depending on your deployment.

If you’re building a web-based voice interface, MindStudio’s agent can output the audio URL directly to your frontend. For phone deployments, you’d route through Twilio or a similar SIP provider.

Handle the Booking Confirmation Step

Before the agent actually books the slot, add a confirmation branch:

1. Agent states the booking details: “Just to confirm — I’m booking a 30-minute call for Sarah Chen at sarah@example.com on Tuesday, March 4th at 2pm Eastern. Does that sound right?”
2. Caller responds yes or no
3. If yes → call Calendly to create the invitee → confirm verbally
4. If no → loop back to slot selection or name/email collection

This confirmation step prevents booking errors and makes the agent feel trustworthy.

---

Where MindStudio Makes This Significantly Easier

Building a voice agent that integrates two external APIs, manages multi-turn conversation state, and handles booking logic from scratch is a real engineering project. MindStudio collapses that into a visual workflow you can build in under an hour.

A few specific things that matter for this use case:

No API key management headaches. MindStudio lets you store your ElevenLabs and Calendly credentials as secure variables, then reference them across your workflow. You don’t expose keys in code or configuration files.

State management across turns. A booking conversation has multiple turns — collecting availability preferences, presenting options, collecting name and email, confirming. MindStudio’s workflow variables persist across turns automatically, so you don’t have to manage session state yourself.

1,000+ pre-built integrations. Beyond ElevenLabs and Calendly, you can extend the agent with one block — log the booking to a Google Sheet, send a Slack notification to your sales team, add the contact to HubSpot, or trigger a follow-up email sequence. All without leaving the builder.

Flexible deployment. Once built, you can expose the agent as a webhook endpoint, embed it in your website, trigger it from an email, or connect it to a phone number via Twilio. The same workflow works across all of these.

You can start building for free at mindstudio.ai. Most booking agents like this take 30–60 minutes to set up once you have your API credentials ready.

If you’re looking for inspiration on what else you can automate with a similar setup, the MindStudio use case library has examples across sales, support, and operations.

---

Testing Your Voice Agent Before Going Live

Don’t skip testing. A broken booking experience is worse than no booking experience — it erodes trust fast.

Test the Calendly API Calls Independently

Before testing the full conversation, verify each API call works on its own:

• Hit the availability endpoint and confirm you’re getting valid time slots back
• Create a test booking manually using the POST endpoint and verify it appears in Calendly
• Check that your event type URI is correct — this is the most common source of errors

Run the Full Conversation End-to-End

Simulate a caller by typing (or speaking) through the full booking flow:

1. Start with an ambiguous opening (“I’m interested in learning more about your service”)
2. Let the agent surface booking intent naturally
3. Select a time slot
4. Provide a test name and email
5. Confirm the booking
6. Verify the appointment appears in your Calendly dashboard

Test Edge Cases

These are the scenarios most likely to cause issues:

• No available slots: What does the agent say when the next 7 days are fully booked?
• Caller changes their mind mid-booking: Can they go back and pick a different slot?
• Invalid email format: Does the agent catch this or send a malformed API request?
• Caller hangs up early: Does the partial conversation cause any errors?

Add handling for each of these in your workflow before launch.

Common Mistakes to Fix

The agent speaks formatted text. If your system prompt doesn’t explicitly say “no bullet points, no markdown,” the LLM will sometimes return formatted responses that sound bizarre when spoken.

Timezone confusion. Calendly returns times in UTC. Make sure your agent either converts to the caller’s timezone or confirms which timezone is being used before booking.

Slot availability goes stale. If you fetch available slots early in the conversation and the caller takes several minutes to decide, that slot might get booked by someone else. Refresh availability before finalizing.

No fallback for failed bookings. If the Calendly API call fails at the last step, the caller should hear a clear message and get an alternative path — not silence or an error code.

---

Deploying the Voice Agent

Once testing is solid, you have several deployment options depending on where callers are coming from.

Web Widget

Embed the voice agent directly on your website — typically on a pricing page, contact page, or demo request page. MindStudio generates an embed snippet you can drop into any site. The caller speaks through their browser mic, and the agent responds in audio.

Phone Number via Twilio

For inbound phone calls, connect your MindStudio agent to a Twilio phone number. Calls route to your agent, which handles the full conversation. This is the most common setup for sales or support teams who want to automate inbound scheduling. Twilio’s documentation covers the voice webhook setup needed to point calls at your agent endpoint.

Outbound Calling

With the same setup, you can flip the flow for outbound — trigger the agent to call a list of leads, qualify them, and book meetings automatically. This requires a bit more configuration (a dialer integration and an outbound trigger workflow) but uses the same core agent logic.

---

Frequently Asked Questions

Can a voice agent actually book Calendly meetings without a human involved?

Yes. Using the Calendly API, a voice agent can programmatically check availability and create a scheduled event — the entire process happens in the conversation, with no human needed. The caller gets a confirmation email from Calendly automatically once the booking is created.

Do I need to know how to code to build this?

No. The setup described in this guide uses MindStudio’s no-code visual builder for workflow logic, ElevenLabs’ API for voice (accessible via HTTP blocks), and Calendly’s standard API. The only “technical” part is working with API keys and HTTP request blocks — both of which are configuration, not code.

What’s the difference between ElevenLabs and other text-to-speech options?

ElevenLabs is known for producing the most natural-sounding AI voices currently available — particularly in tone variation, pacing, and emotional nuance. Alternatives like Google Cloud TTS, Amazon Polly, and Microsoft Azure TTS are cheaper and widely used, but most sound more robotic. For a booking agent where trust matters, the quality difference is noticeable. That said, any of these can be swapped into the same architecture.

Which Calendly plan do I need for this?

Calendly’s Standard plan ($10/seat/month) gives you API access for reading availability. Programmatic booking creation (without using a hosted Calendly page) requires the Teams plan or higher. If you’re on Standard, you can still build a functional agent — it presents available slots and provides a pre-filled Calendly link, rather than booking directly via API.

Can the voice agent handle objections or questions before booking?

Yes, and this is one of the strengths of building on a language model rather than a decision tree. You can give the agent knowledge about your product, pricing, FAQs, or objection responses in its system prompt. It handles those questions in natural conversation before surfacing the booking option. This makes it useful as both a qualification tool and a scheduling tool.

How do I handle callers in different time zones?

The cleanest approach is to ask the caller their timezone early in the conversation (“Just so I give you accurate times — what timezone are you in?”) and store that value as a variable. Then convert the UTC times returned by Calendly into that timezone before presenting options. Most language models can handle simple timezone math if you pass the raw UTC time and the desired timezone as context.

---

Key Takeaways

• A voice agent using ElevenLabs and Calendly can fully automate meeting scheduling — from greeting to confirmed booking — without any human involvement.
• The core architecture is: language model for conversation logic, ElevenLabs for voice, Calendly API for availability and booking, and a workflow tool (like MindStudio) to connect all three.
• Your system prompt should explicitly avoid formatted output — spoken responses need to sound natural, not like a document.
• Test edge cases before launch: no availability, mid-conversation changes, timezone handling, and API failure fallbacks.
• Deployment options include web embedding, inbound phone (via Twilio), and outbound calling — all from the same underlying agent workflow.

If you want to build this without managing infrastructure, authentication, or API plumbing yourself, MindStudio gives you the visual builder, the integrations, and the model access to get it running fast. The average build for something like this takes under an hour.