Breeze TTS 2: Specs, VRAM Needs, and Local Setup Guide
Breeze TTS 2's specs: sub-40ms latency, 12GB minimum VRAM, voice cloning and design features, and how to run it locally.
What is Breeze TTS 2?
Breeze TTS 2 is an open-weight text-to-speech model from BreezeBlue, released with PyTorch inference code and full model weights on Hugging Face. It’s built for real-time voice applications: sub-40 millisecond time to first audio, streaming output, and three distinct ways to control a voice (cloning, natural-language design, and directed delivery). BreezeBlue reports it ranks first among open-weight models on the Artificial Analysis TTS leaderboard and beats some closed, proprietary systems on that same benchmark.
TL;DR
- Breeze TTS 2 is an open-weight bilingual (English and Chinese) text-to-speech model with publicly released weights and Apache 2.0 licensed inference code.
- It hits under 40ms time to first audio on an NVIDIA H100 using its warmed-up fast path, which matters for anything conversational like voice agents or live narration.
- Streaming generation runs at a 0.32 real-time factor, meaning it produces audio roughly 3.1 times faster than real playback speed once warmed up.
- Minimum hardware is a 12GB GPU for standard eager inference (about 7.7 GiB actual usage), while the fastest CUDA-graph-accelerated path needs a 24GB GPU (about 14.4 GiB).
- The model supports three control modes: voice cloning from reference audio and transcript, voice design from a text description with no reference audio, and voice direction, which clones a speaker’s identity while steering tone and pace.
- It also supports inline vocal events like laughs, coughs, and sighs, written directly into the input text using parentheses in English or brackets in Chinese.
- Model weights carry a research and non-commercial license; commercial use requires separate written authorization from RESONIA, INC., even though the inference code itself is Apache 2.0.
Remy doesn't build the plumbing. It inherits it.
Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something.
Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want.
What are Breeze TTS 2’s technical specs?
The headline numbers, per BreezeBlue’s own model card, are latency and streaming throughput. On an NVIDIA H100 with the “fast path” warmed up, Breeze TTS 2 achieves under 40ms time to first audio (TTFA), the delay between sending text and getting the first chunk of speech back. That’s fast enough to feel instantaneous in a live conversation.
Streaming throughput comes in at a 0.32 real-time factor (RTF) on the same warmed-up H100 configuration, which BreezeBlue translates to about 3.1x real time. In practice that means the model can generate audio for a given stretch of speech in roughly a third of the time it takes to actually play that speech back, which leaves comfortable headroom for network overhead in a live pipeline.
On memory, eager inference (the default, non-optimized mode) uses approximately 7.7 GiB of GPU memory. Enabling the full fast path (--fast-all, which turns on CUDA Graph optimizations across every inference stage) pushes memory to about 14.4 GiB. BreezeBlue’s official minimums are a 12GB GPU for eager inference and a 24GB GPU if you want the fast path enabled.
The model is bilingual out of the box, generating English and Chinese speech from a single checkpoint, and ships as safetensors weights compatible with the Hugging Face transformers library, alongside a dedicated audio tokenizer component.
How does the voice control system work?
Breeze TTS 2 splits voice control into three separate modes rather than treating TTS as a single “type text, get audio” pipeline.
Voice clone takes a reference audio clip plus its exact transcript and reproduces the timbre, rhythm, emotion, and style of that speaker for new text. This is the standard cloning approach: the model needs to know precisely what was said in the reference so it can isolate how it was said.
Voice design goes the opposite direction: no reference audio at all. Instead you supply a natural-language description, like “a warm, thoughtful young woman with a clear voice and a calm, reflective delivery,” and the model generates a voice matching that description from scratch. BreezeBlue recommends setting --cfg-scale 4 to make the model follow the instruction more closely.
Voice direction is a hybrid: you provide reference audio and transcript to lock in a specific speaker’s identity, but also add an instruction telling the model how to deliver the new text, for example “speak slowly with a restrained, serious tone.” This separates who is speaking from how they’re speaking, which is useful if you want a consistent character voice that can still shift emotional register scene to scene.
All three modes accept inline vocal events embedded directly in the text: (laugh), (cough), (clears throat), and (sigh) in English, or their bracketed Chinese equivalents like [笑] and [叹气]. These get rendered as actual non-speech sounds in the output audio rather than being read aloud as words.
How do you run Breeze TTS 2 locally?
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
Setup follows a fairly standard open-source ML workflow. You need Linux, Python 3.10 or newer, and a CUDA-capable NVIDIA GPU. The steps:
- Clone the inference repository from GitHub (
breezeblue-ai/breeze-tts). - Install dependencies with
pip install -r requirements.txt. - Download the Breeze TTS 2 checkpoint, which bundles all required model components including the audio tokenizer.
- Optionally build the provided Docker image for a tested CUDA environment. The default image targets H100/Hopper (sm90) GPUs; for A100 you set
FLASH_ATTN_CUDA_ARCHS=80before building.
From there, inference runs through a single script, infer.py, pointed at the checkpoint directory with flags for reference audio, reference transcript, target text, and optional instruction and CFG scale. The same script handles all three voice modes; which one runs depends on which combination of flags you pass.
For anything resembling a production or app-like use case, BreezeBlue also ships a streaming API. Running python -m breeze_infer.api starts a single-concurrency server that accepts POST requests and streams back mono 24kHz, 16-bit PCM audio. This is the same runtime used by the CLI, just wrapped in an HTTP interface, and it supports the same reference audio, instruction, and CFG scale parameters via multipart form fields.
Is the fast path worth enabling?
That depends on your deployment shape. By default, both the CLI and the API run in eager mode: no graph warmup, lower memory footprint, but slower per-request latency. The --fast-all flag turns on CUDA Graph optimization across five separate inference stages: text encoding, backbone prefill, backbone decode, the depth decoder, and the audio codec. Each stage can also be toggled independently with its own flag, which is mainly useful for profiling or isolating a bottleneck rather than for typical deployment.
The tradeoff is upfront cost: enabling the fast path adds cold-start warmup time and roughly doubles GPU memory use (7.7 GiB to 14.4 GiB), which is what pushes the hardware requirement from a 12GB card to a 24GB card. If you’re running a long-lived server process handling many requests, that one-time warmup cost amortizes quickly and the sub-40ms TTFA number becomes achievable. If you’re doing one-off or infrequent generation, eager mode avoids the memory overhead and warmup delay at the cost of slower individual responses.
Can you use Breeze TTS 2 commercially?
Not without separate permission. The inference code on GitHub is licensed under Apache 2.0, which is permissive, but that license explicitly does not extend to the model weights. The weights, checkpoints, adapters, and any derivative models or self-hosted outputs fall under BreezeBlue’s own Research and Non-Commercial License. Commercial use requires written authorization from RESONIA, INC., the entity behind BreezeBlue, and hosted BreezeBlue services run under their own separate terms. Anyone deploying this for a commercial product needs to contact BreezeBlue directly rather than relying on the open weights alone.
Frequently Asked Questions
What GPU do I need to run Breeze TTS 2?
A 12GB GPU is the minimum for standard eager inference, which uses about 7.7 GiB of VRAM. If you want the fully accelerated fast path with CUDA Graphs enabled across all inference stages, you need a 24GB GPU, since that configuration uses roughly 14.4 GiB.
Plans first. Then code.
Remy writes the spec, manages the build, and ships the app.
How fast is Breeze TTS 2 compared to real-time speech?
On a warmed-up NVIDIA H100 using the fast path, it reaches a 0.32 real-time factor, meaning it generates audio about 3.1 times faster than it takes to play back. Time to first audio drops under 40 milliseconds in that same configuration.
Does Breeze TTS 2 support languages other than English?
Yes. It’s a bilingual model that generates both English and Chinese speech from the same checkpoint, and it supports inline vocal events in both languages using different bracket conventions (parentheses for English, square brackets for Chinese).
Is Breeze TTS 2 free to use?
The inference code is open source under Apache 2.0, and the model weights are publicly downloadable, but the weights carry a non-commercial research license. Commercial deployment requires a separate written license from RESONIA, INC.
What’s the difference between voice cloning and voice direction in Breeze TTS 2?
Voice cloning reproduces a reference speaker’s timbre and style as-is from an audio sample and its transcript. Voice direction also clones the reference speaker’s identity but adds a natural-language instruction that steers tone, emotion, and pace, letting you keep the same voice while changing how it delivers a specific line.
