How to Run Fish Audio's Drama 3 Locally for Voice Cloning
A practical guide to installing Fish Audio's Drama 3 voice cloning model locally, covering VRAM needs, setup steps, and consent verification.
What is Fish Audio’s Drama 3?
Drama 3 is an open-weight text-to-speech model from Fish Audio built for voice cloning, multi-speaker dialogue, and expressive delivery. It uses a mixture-of-experts text backbone with 35 billion total parameters, only 3 billion of which activate per token, which keeps inference faster and lighter than a dense model of the same size would be. The model supports up to 16 distinct speakers inside a single text input using speaker tags, plus inline direction tags like “whispers,” “laughs,” or “sighs” that change delivery exactly where they’re placed in the script. According to the model card, it can produce first audio output in under 100 milliseconds on a single stream.
TL;DR
- Drama 3 is a sparse mixture-of-experts TTS model (35B total parameters, 3B active per token) from Fish Audio, hosted on Hugging Face as
fishaudio/drama-3. - It supports voice cloning, multi-speaker scripts with up to 16 characters, and inline emotion or delivery tags like whisper, sigh, or laugh.
- Running it locally requires serious hardware: the setup demonstrated used an H100 GPU with 80GB of VRAM, and Fish Audio recommends Hopper or Ada class cards.
- Installation is a short three-step process: clone the repo, install prerequisites, and serve it locally (one tester used an HTTP-based serving setup).
- Before cloning any voice, the model enforces a consent gate: you read a generated sentence aloud, and the system checks it against your voice (one test showed a 93% match) before cloning proceeds.
- Real-time generations are noticeably rougher than the curated demo clips on the Hugging Face page, and outputs can be inconsistent, sometimes requiring a regenerate.
- The model card lists streaming and multi-speaker API support, but a few features, like continuous editing, don’t appear to be available in the open release.
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
How does Drama 3’s architecture work?
Drama 3 uses a mixture-of-experts (MoE) design for its text backbone. Instead of running every parameter for every token, the model routes each token through a small subset of “expert” subnetworks. That’s why it has 35 billion parameters on paper but only activates about 3 billion per token during inference. This is the same general approach used in large MoE language models: you get the capacity of a much bigger dense model without paying its full compute cost on every forward pass.
The Hugging Face repository lists the model under the fish_omni architecture tag, with weights split across eight safetensors shards plus a separate codec file (codec.pth) used for audio tokenization. The repo also declares support for dozens of languages, from major ones like English, Chinese, Japanese, Korean, and Spanish down to lower-resource languages like Welsh, Basque, and Yiddish, suggesting a broad multilingual training set even if real-world quality will vary language by language.
What hardware do you need to run Drama 3 locally?
This is the practical bottleneck. Fish Audio’s documentation calls for a Hopper or Ada generation GPU, the same architecture tier as Nvidia’s H100 and L40S/4090-class cards. A real-world test of local inference ran on an H100 with 80GB of VRAM, and that was treated as a baseline rather than overkill. If you don’t already own a card in that class, renting GPU time from a cloud provider is the realistic path, since consumer GPUs with less VRAM are unlikely to load the full model comfortably.
Beyond VRAM, budget time and disk space for the download itself. With eight safetensors shards covering a 35-billion-parameter MoE model, the total footprint is large, so a fast connection and adequate storage matter before you even get to installation.
How do you install and serve Drama 3?
The core installation workflow is short at a high level:
- Clone the repository from Hugging Face or Fish Audio’s official source.
- Install the prerequisites, which includes the Python environment and dependencies the model needs to run.
- Serve the model locally over HTTP so you can send text and reference audio to it and get generated speech back.
That’s the shape of it: three stages, no exotic orchestration required, but each step assumes you’re comfortable with a command line, Python package management, and basic GPU server setup. Given the VRAM requirement, most people will run this on a rented cloud GPU instance rather than a personal workstation, in which case you’re also handling SSH access and port forwarding to reach the local server from your own machine.
How does the voice consent system work?
Seven tools to build an app. Or just Remy.
Editor, preview, AI agents, deploy — all in one tab. Nothing to install.
Before Drama 3 will clone a voice, it runs you through a consent gate, a safeguard that only allows cloning after the speaker explicitly agrees. The flow works like this: the system generates a specific consent sentence, you record yourself reading it aloud, and the model checks that recording against a reference sample of your voice. In one demonstrated run, the match came back at 93%, a score that the creator noted will vary model to model and attempt to attempt. Once verification passes, the system proceeds to transcribe your sample via speech recognition, and from there you can feed in new text for it to generate in your cloned voice.
This matters because open voice cloning models have an obvious misuse path: cloning someone’s voice without their knowledge. Building a verification step directly into the pipeline, rather than leaving it as an optional add-on, is a meaningful design choice, even if it’s not a perfect guarantee against bad actors working around the official tooling.
Is Drama 3 good enough for real use?
The honest answer, based on hands-on testing, is “promising but inconsistent.” The model handles some things well: paired direction tags noticeably change delivery exactly where they’re placed, so a word wrapped in a whisper tag comes out quieter and a phrase marked for emphasis gets stressed correctly, all within a single generation and in the cloned voice. Suspenseful and nervous line readings came across as reasonably convincing in testing, described as roughly 60% successful, with the rest needing another take.
Where it falls short is consistency. A real-time generation attempting fast, breathless sports commentary with a shout and a laugh of relief came out flat compared to what the direction called for. And when compared head to head, the curated example clips published on Fish Audio’s own Hugging Face page sound noticeably more polished than what you get generating fresh, unedited output locally. That gap between “best curated demo” and “typical real output” is worth keeping in mind before assuming the model is production ready. It also doesn’t currently expose some advertised capabilities, like continuous editing, in the freely available version.
Who should try running Drama 3 locally?
Developers and researchers interested in open TTS and voice cloning, people building character-driven audio content, or anyone experimenting with multi-speaker dialogue synthesis are the natural audience. It is less suited, at least right now, to anyone needing dependable, broadcast-ready output on the first try, since regenerating takes is part of the normal workflow. The GPU requirement also filters out casual hobbyists without access to high-VRAM hardware or a cloud rental budget.
Frequently Asked Questions
What is Fish Audio Drama 3 used for?
It’s a text-to-speech model for voice cloning, multi-character audio dialogue (up to 16 speakers in one script), and expressive narration using inline direction tags for things like whispering, sighing, or emphasis.
How much VRAM does Drama 3 need?
Testing was done on an H100 GPU with 80GB of VRAM, and Fish Audio’s documentation recommends Hopper or Ada class GPUs. There’s no confirmed lower bound below that tier, so plan for high-end hardware or a rented cloud GPU.
Does Drama 3 require consent to clone a voice?
Yes. It includes a built-in consent gate: you read a system-generated sentence aloud, and your recording is checked against a reference sample before cloning is allowed to proceed.
Is Drama 3 better than its demo clips suggest?
Not quite the reverse: the official demo clips on Hugging Face sound more polished than typical real-time, unedited generations, so expect some variance and occasional regeneration when using it yourself.
How many parameters does Drama 3 have?
It has 35 billion total parameters, but only about 3 billion are active per token thanks to its mixture-of-experts text backbone, which keeps inference more efficient than a fully dense model of equivalent size.