PhoneLLM Cost Per Minute: The Real Economics of Voice Agent LLMs
PhoneLLM's self-hosted cost-per-minute economics on B200 GPUs, benchmarked against API-based voice agent LLMs like GPT 5.6 Terra.
What does PhoneLLM cost per minute to run?
PhoneLLM, an open-weights model from the Pipecat team at Daily, costs about $0.00025 per minute of voice agent runtime when self-hosted on an NVIDIA B200 GPU at high concurrency. That figure comes from a documented benchmark: 88 concurrent agent sessions per B200 node, an effective GPU cost of $0.2232 per minute after region pinning and utilization targeting, divided across those sessions. Pipecat’s own comparison puts PhoneLLM at roughly 94% cheaper than GPT 5.6 Terra for equivalent voice agent workloads, with faster response times too.
TL;DR
- PhoneLLM is a fine-tuned Nemotron 3 Nano, a 30B-parameter Mixture-of-Experts model with only 3.5B active parameters, which is why it can run cheaply and quickly on a single GPU.
- The headline number is $0.00025 per minute per agent when running 88 concurrent sessions on a single B200, based on Modal’s published GPU pricing and a 70% utilization target.
- Pipecat claims PhoneLLM matches GPT 5.6 Terra on accuracy while costing about 94% less and delivering a 1,300ms faster P95 time-to-first-token.
- Concurrency is the whole game: cost per minute drops sharply as more agent sessions share a GPU, but pushing concurrency too high blows through latency budgets, so the real optimization is finding the maximum concurrency that still hits sub-600ms P95 response time.
- Self-hosting only saves money when the model is right-sized for the task; large API providers already operate at massive scale, so the savings come from using a smaller specialized model instead of a general-purpose frontier model.
- PhoneBench v1, Pipecat’s companion benchmark, scores models on accuracy, tool-call reliability, latency, and imputed cost per minute, giving a standardized way to compare voice agent LLMs beyond raw benchmark accuracy.
- The model is released under a BSD license with no commercial restrictions, and can be deployed via vLLM, SGLang, or Modal’s Auto Endpoints.
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
Why does cost per minute matter more than token pricing for voice agents?
Most LLM APIs price by the token: separate rates for input, output, cached reads, cached writes, and sometimes a premium “fast” service tier. That pricing model works fine for batch jobs or chat apps where a user waits for a response. It breaks down for voice agents, which run continuously across many turns of a live phone call.
A voice agent’s economics depend on how long calls last, how big the system prompt is, how much the model “thinks” before answering, and how much of the context gets cached across turns. Converting all of that into a single cost-per-minute number is the only way to compare apples to apples across models and hosting setups. It’s also the number that actually maps to a business’s unit economics: cost per call, cost per booked appointment, cost per resolved support ticket.
Pipecat built a public spreadsheet modeling this conversion for a range of models, both API-based and self-hosted, using the interaction patterns observed in PhoneBench. The pattern that falls out: bigger, more general-purpose models cost more per minute, and models with aggressive “thinking” behavior (even set to a minimal thinking mode) can rack up hidden token costs. Gemini 3.6 Flash is called out as one such outlier, generating substantial thinking tokens even when reasoning is nominally turned down.
How does self-hosting actually get cheaper than an API?
The naive assumption is that running your own GPU is always cheaper than paying an API provider. That’s not really true. Large inference providers operate at scale efficiencies that are hard to beat with a single rented GPU. The place self-hosting wins is what Pipecat calls “model size arbitrage”: swapping a large, general-purpose frontier model for a smaller model fine-tuned tightly for the actual task.
A phone agent doesn’t need the full breadth of a frontier model’s world knowledge or reasoning depth. It needs to hold a conversation, call the right tools at the right time, and do it fast. PhoneLLM is trained specifically for that narrower job, on top of Nemotron 3 Nano’s Mixture-of-Experts architecture. With only 3.5B of its 30B parameters active per token, it can serve many concurrent conversations on modest hardware without the latency or cost overhead of a dense, much larger model.
The other lever is inference-stack control. Because the serving code (vLLM, SGLang) and the weights are both open, teams can tune the exact point on the cost-latency curve that fits their use case, and even change the curve itself by writing custom serving logic. Pipecat notes it has done this both for LLM serving and for NVIDIA’s transcription models.
How is the $0.00025 per minute figure calculated?
The number comes from a concurrency benchmarking sweep, not a theoretical estimate. Pipecat ran simulations to find the maximum number of concurrent agent processes a single GPU could handle while keeping P95 time-to-first-answer-token under a 600ms “not to exceed” target.
For PhoneLLM (and the underlying Nemotron 3 Nano architecture), that maximum was 44 processes per GPU instance, which translates to 88 agent processes pinned to each B200 node under Pipecat’s benchmark setup.
From there, the cost math uses Modal’s published B200 pricing:
- Base cost: $6.2496 per hour
- With a 1.5x region-pinning multiplier: $9.3744 per hour
- Adjusted for a 70% utilization target (dividing by 0.70): $13.392 per hour, or $0.2232 per minute
- Divided across 88 concurrent agents: $0.00025 per minute per agent
That last step is the crux of the whole economic story: the GPU’s cost doesn’t change much whether it’s running 10 agents or 88, so packing more concurrent sessions onto the same hardware is what drives the per-minute cost down. The constraint is that packing too many sessions onto one GPU increases latency, and voice agents have a hard requirement on how slow they can be before callers notice and get frustrated.
Why does latency limit how cheap a voice agent can get?
Voice conversations have a tight tolerance for delay. Pipecat’s data points to roughly 1,500ms as the threshold for acceptable P95 voice-to-voice latency, the time from when a caller stops speaking to when they hear a response start. That total budget has to cover network transport, audio processing, speech-to-text, the LLM’s own response generation, and text-to-speech, all before the caller hears anything.
In a well-optimized pipeline, Pipecat allocates about 650ms of that budget specifically to LLM time-to-first-token. That’s a tight window. For comparison, GPT 5.6 Terra’s P95 time-to-first-token in its fast mode was measured at about 1,900ms, already blowing past the entire voice-to-voice budget before accounting for STT or TTS.
This is why cost and latency can’t be optimized independently. Cramming more concurrent requests onto a GPU lowers cost per minute but increases queuing delay, which pushes time-to-first-token up. Pipecat’s benchmarking approach sets a latency ceiling first (sub-600ms P95 TTFAT) and then finds the maximum concurrency that respects it. That’s a more defensible way to report cost than simply maximizing throughput and ignoring what it does to response time.
Is PhoneLLM worth it compared to an API-based model?
For teams running voice agents at meaningful volume, the tradeoff comes down to control versus convenience. PhoneLLM requires self-hosting: deploying on infrastructure that can run a 30B-parameter Mixture-of-Experts model, using vLLM or SGLang with the Nemotron 3 Nano serving recipes, or using Modal’s Auto Endpoints for a managed deployment path. That’s more operational overhead than calling an API.
In exchange, Pipecat’s benchmark data shows PhoneLLM matching or exceeding the accuracy of commonly used production voice agent models on PhoneBench, while cutting cost by roughly 94% versus GPT 5.6 Terra and cutting P95 time-to-first-token by about 1,300ms. Because the model is BSD-licensed with no commercial restrictions, there’s no licensing cost layered on top of compute.
The catch is that these numbers assume a team can hit the concurrency levels needed to realize the savings. At low volume, with a single GPU running only a handful of concurrent calls, the per-minute cost advantage shrinks because the fixed GPU cost isn’t being spread across as many sessions. The economics favor teams with enough call volume to keep a GPU cluster consistently busy.
Plans first. Then code.
Remy writes the spec, manages the build, and ships the app.
Frequently Asked Questions
What is PhoneLLM?
PhoneLLM Alpha 1 is an open-weights language model built by the Pipecat team at Daily, fine-tuned from NVIDIA’s Nemotron 3 Nano 30B-A3B model specifically for voice agent tasks like phone-based customer service and outbound calling.
How much cheaper is PhoneLLM than GPT 5.6 Terra?
Pipecat’s benchmark data puts PhoneLLM at approximately 94% cheaper per minute than GPT 5.6 Terra for comparable voice agent workloads, while also delivering a P95 time-to-first-token about 1,300ms faster.
What hardware does PhoneLLM run on?
It’s designed to run on a single NVIDIA B200 GPU, using vLLM or SGLang for serving, or via Modal’s Auto Endpoints for managed deployment. It fits comfortably on one B200 with room left for long context and caching.
What is PhoneBench?
PhoneBench v1 is a companion benchmark from Pipecat that evaluates LLMs for phone agent suitability, scoring accuracy, tool-call reliability, speaking style, latency, and an estimated cost per minute using a panel of LLM judges calibrated against human labels.
Does self-hosting always cost less than an API?
No. Large API providers benefit from massive scale efficiencies. Self-hosting only becomes cheaper when a smaller, task-specific model like PhoneLLM replaces a larger general-purpose model, and when concurrency on the GPU is high enough to spread fixed hardware costs across many simultaneous sessions.