How to Run Breeze TTS 2 Locally: GPU Requirements and Setup
Step-by-step guide to self-hosting Breeze TTS 2, covering GPU memory needs, Docker builds, and voice clone, design, and direction commands.
What is Breeze TTS 2?
Breeze TTS 2 is an open-weight text-to-speech model from BreezeBlue, built for real-time voice interaction rather than offline batch narration. It runs on a single consumer or datacenter GPU, supports English and Chinese in one checkpoint, and adds three distinct modes: cloning a voice from reference audio, designing a voice from a text description with no reference audio at all, and directing an existing cloned voice toward a specific tone or pace. According to BreezeBlue, it ranks first among open-weight models on the Artificial Analysis TTS leaderboard and beats some closed, proprietary systems on the same benchmark.
TL;DR
- Minimum hardware for Breeze TTS 2 is a 12 GB CUDA-capable NVIDIA GPU running Linux with Python 3.10 or newer, since eager inference uses about 7.7 GiB of VRAM.
- The fast inference path (CUDA Graphs across every stage) needs roughly 14.4 GiB of GPU memory, so BreezeBlue recommends a 24 GB card if you want the low-latency mode.
- Setup is a git clone plus pip install, with an optional Docker build script that targets H100/Hopper GPUs by default and A100 GPUs with one environment variable override.
- Three inference modes cover different needs: voice clone for preserving an exact speaker’s timbre, voice design for generating a new voice from a natural-language description, and voice direction for steering a cloned voice’s emotion and pace.
- Latency numbers are aggressive on top-tier hardware: under 40 ms time-to-first-audio and a 0.32 real-time factor (about 3.1x real time) on a warmed-up H100, though those figures assume the fast path, not the default eager mode.
- The license is research and non-commercial only. The inference code is Apache 2.0, but model weights and any self-hosted output fall under a separate BreezeBlue license that blocks commercial use without written permission.
- Vocal events are supported inline, so you can drop cues like
(laugh)or(sigh)in English text, or[笑]and[叹气]in Chinese, directly into the script you feed the model.
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
What GPU do you need to run Breeze TTS 2?
The floor is a CUDA-capable NVIDIA GPU with at least 12 GB of VRAM. That covers the default “eager” inference mode, which BreezeBlue measures at approximately 7.7 GiB of GPU memory in practice. Eager mode skips graph warmup and CUDA Graph compilation, so it starts faster but runs slower per request.
If you want the low-latency streaming performance Breeze TTS 2 is designed for, you need the --fast-all flag, which turns on CUDA Graph optimizations across the text encoder, backbone prefill, backbone decode, depth decoder, and codec stages. That configuration uses about 14.4 GiB of VRAM, which is why BreezeBlue recommends a 24 GB GPU for the fast path. In practice this means a 12 GB card (something like an RTX 3060 12GB or 4070 Ti) gets you a working local install, but a 24 GB card (RTX 3090, 4090, or A100/H100-class hardware) is what enables the sub-40ms time-to-first-audio numbers BreezeBlue reports on an H100.
The system also needs Linux and Python 3.10 or newer. There’s no documented Windows-native path or CPU-only fallback in the model card, so this is a GPU-and-Linux project by design.
How do you install and run Breeze TTS 2?
Setup follows a standard PyTorch inference project pattern. Clone the repository, install dependencies, and download the checkpoint:
git clone https://github.com/breezeblue-ai/breeze-tts.git
cd breeze-tts
python -m pip install -r requirements.txtThe Breeze TTS 2 checkpoint bundles all required model components, so there’s no separate vocoder or tokenizer download to hunt down.
For a reproducible environment, BreezeBlue ships a Docker build script tuned for the CUDA setup they test against:
bash docker/build.shBy default this targets H100/Hopper-class GPUs (compute capability sm90). If you’re on an A100 instead, override the CUDA architecture flag before building:
FLASH_ATTN_CUDA_ARCHS=80 bash docker/build.shThat single environment variable swap matters because the image bundles FlashAttention, which needs to be compiled for the specific GPU architecture it will run on. Building for the wrong architecture either fails outright or silently falls back to slower attention kernels.
How do voice clone, voice design, and voice direction differ?
These are the three core commands, and picking the right one depends on what you’re trying to produce.
Voice clone takes a reference audio clip and its exact transcript, then generates new speech in that same voice. It’s invoked with --ref-audio and --ref-text flags alongside the target --text:
python infer.py ../breeze-tts-2 \
--ref-audio reference_en.wav \
--ref-text "This is the exact transcript of the English reference audio." \
--text "(sigh) It is good to hear your voice again after all this time." \
--output outputs/voice_clone_en.wavBreezeBlue notes the reference transcript needs to match the audio exactly, and the source audio should be clean with minimal background noise. This mode preserves timbre, rhythm, emotion, and style from the reference.
Voice design skips reference audio entirely. Instead you describe the voice you want in natural language via --instruction, and the model generates a voice matching that description:
python infer.py ../breeze-tts-2 \
--text "(sigh) Welcome aboard. Your journey begins now." \
--instruction "A warm, thoughtful young woman with a clear voice and a calm, reflective delivery." \
--cfg-scale 4 \
--output outputs/voice_design_en.wavThe --cfg-scale 4 flag strengthens how closely the model follows the instruction text, and BreezeBlue recommends it for both voice design and voice direction.
Voice direction combines the two: it clones an identity from reference audio, but overrides delivery using an instruction, letting you keep a consistent speaker while changing tone, pace, or emotional register on a line-by-line basis. This is the mode most relevant for things like game dialogue or narration where you want one voice actor but many emotional deliveries.
Is the streaming API worth setting up?
If you’re building a real-time voice application rather than generating files offline, yes. BreezeBlue includes a single-concurrency streaming server:
python -m breeze_infer.api ../breeze-tts-2 --host 0.0.0.0 --port 7860It runs the same PyTorch runtime and defaults to eager execution, but you can start it with --fast-all to enable the CUDA Graph fast path for lower latency. Requests go through a standard multipart POST:
curl -X POST http://127.0.0.1:7860/v1/audio/speech \
-F "cfg_scale=4" \
-F "ref_audio=@reference.wav" \
-F "ref_text=This is the exact transcript of the reference audio." \
-F "text=(clears throat) We need to discuss what happened last night." \
-F "instruction=Speak slowly with a restrained, serious tone." \
-F "seed=42" \
--output voice_direction.pcmThe response streams as mono 24 kHz signed 16-bit PCM, which is a common format for feeding directly into audio players or downstream processing without additional decoding steps. The “single-concurrency” label is worth noting: this server handles one request at a time rather than batching multiple simultaneous users, so a production deployment serving many users concurrently would need to run multiple instances behind a load balancer.
Each of the five inference stages (text encoder, backbone prefill, backbone decode, depth decoder, codec) can also be toggled independently with flags like --fast-text-encoder or --no-fast-codec, which BreezeBlue says is intended mainly for profiling and debugging rather than typical use.
What’s the license situation for self-hosted Breeze TTS 2?
This is the part worth reading carefully before building anything real on it. The inference code itself is Apache 2.0, a permissive open-source license. But the model weights, any derivative models, and outputs generated by self-hosting Breeze TTS 2 fall under a separate BreezeBlue Research and Non-Commercial License. The Apache license on the code does not extend commercial rights to the model.
In plain terms: you can freely read, modify, and redistribute the inference code, but you cannot use the weights, fine-tunes, or generated audio in a commercial product without written authorization from RESONIA, INC., the entity BreezeBlue names for commercial licensing inquiries. Hosted versions of the service, if BreezeBlue offers them, are governed by separate terms.
The model card also places responsibility on the user for obtaining consent for any reference audio or voices used with the cloning and direction features, and explicitly prohibits unauthorized voice cloning, impersonation, and fraud.
Frequently Asked Questions
What is the minimum GPU memory to run Breeze TTS 2?
About 12 GB of VRAM for default eager inference, which measures around 7.7 GiB in actual use. The fast inference path with CUDA Graphs enabled needs roughly 14.4 GiB, so a 24 GB GPU is recommended if you want that mode.
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
Does Breeze TTS 2 run on Windows or CPU-only machines?
The documented requirements specify Linux and a CUDA-capable NVIDIA GPU. There’s no CPU-only inference path or native Windows support described in the setup instructions.
Can I use Breeze TTS 2 output commercially?
Not without written authorization. The model weights and any self-hosted outputs are licensed for research and non-commercial use only, separate from the Apache 2.0 license covering the inference code itself. Commercial use requires contacting RESONIA, INC.
What’s the difference between voice clone and voice direction?
Voice clone reproduces a reference speaker’s voice as closely as possible using their audio and matching transcript. Voice direction also uses reference audio to preserve the speaker’s identity, but adds a natural-language instruction to change delivery, such as speaking slower or with a more serious tone.
How fast is Breeze TTS 2 at generating audio?
BreezeBlue reports under 40 ms time-to-first-audio and a 0.32 real-time factor (about 3.1x real-time generation speed) using the warmed-up fast path on an NVIDIA H100. Performance on smaller GPUs or in default eager mode will be slower.
