How to Run Carbon-X Locally for DNA Analysis on One GPU
A setup guide for self-hosting Carbon-X, the 14.8B DNA language model, on a single 80GB GPU using vLLM or Transformers for local inference.
What is Carbon-X and why run it locally?
Carbon-X is a 14.8-billion-parameter open-weight language model from Hugging Face Bio that reads and writes DNA sequences inside normal chat conversations. It predicts whether mutations are benign or pathogenic, flags regulatory regions, classifies sequences by organism, and predicts upcoming bases, all while explaining its answers in plain English. Because the full model fits in bfloat16 on a single 80GB GPU, labs and researchers can run it entirely on their own hardware instead of sending unpublished genomic data to a third-party API.
TL;DR
- Carbon-X runs on one 80GB GPU in bfloat16, which means sensitive or unpublished DNA sequences never have to leave your own infrastructure.
- The model averages 66.7% across five DNA benchmarks with thinking disabled, nearly matching GPT-6.1 Sol (67.5%) while costing roughly 1/1,800th as much per problem.
- It uses a hybrid tokenizer: standard BPE for English text and non-overlapping 6-mers for DNA, unified in a single token stream with no separate DNA encoder.
- You need Transformers v5 or later and
trust_remote_code=Truewhen loading the tokenizer, since the DNA tokenization logic is custom code. - vLLM is the recommended serving path for production use, exposing an OpenAI-compatible endpoint with one command.
- Thinking should stay disabled (
enable_thinking=False), since that’s how the model was post-trained and benchmarked, and it keeps output token counts (and cost) low. - DNA tagging is automatic, but only A/C/G/T form full-resolution 6-mers; ambiguity codes like N can degrade token efficiency if not filtered first.
Seven tools to build an app. Or just Remy.
Editor, preview, AI agents, deploy — all in one tab. Nothing to install.
What hardware do you actually need?
Carbon-X’s weights fit on a single 80GB GPU (an H100 or H200-class card) in bfloat16 precision. That’s the headline spec in the model card, and it’s what makes local deployment realistic for a university lab or a biotech startup rather than something that requires a multi-node cluster. The model card’s own cost modeling uses an H200 at $5/hour as its reference point for throughput, so that’s a reasonable baseline to plan around if you’re estimating compute budgets.
There’s no official quantized release mentioned in the documentation, so if you’re working with a smaller GPU (40GB or less), you’d be quantizing it yourself and should expect some accuracy loss on the benchmarks below. For most self-hosting scenarios aimed at keeping genomic data private, the single 80GB card in full bfloat16 is the tested, supported path.
How do you set up Carbon-X with Transformers?
The basic Python setup mirrors any modern Hugging Face causal language model, with two specifics worth calling out: you need Transformers v5 or later, and the tokenizer requires trust_remote_code=True because Carbon-X ships custom tokenizer code to handle the DNA 6-mer logic.
pip install -U transformersimport torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "HuggingFaceBio/Carbon-X"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda")From there, you build a chat message containing both the reference and variant sequences (or just a raw sequence and a question), apply the chat template with enable_thinking=False, and call generate(). The model’s generation_config.json sets sampling defaults of temperature 0.6, top-p 0.95, and top-k 20, which are also the exact settings used to produce every benchmark number in the model card. If you change them, expect results to drift from published performance.
One operational detail to plan for: on first load, the tokenizer needs Hub access to download the pinned BPE vocabulary it’s built on. If you’re running in a fully air-gapped environment, you’ll need to cache that vocabulary locally ahead of time.
How do you serve Carbon-X with vLLM?
For anything beyond single-shot experimentation, vLLM is the documented path for production serving. It turns Carbon-X into an OpenAI-compatible API endpoint running locally:
pip install -U vllm
vllm serve HuggingFaceBio/Carbon-X --trust-remote-codeOnce the server is up, you query it like any OpenAI-style chat endpoint:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="HuggingFaceBio/Carbon-X",
messages=[{"role": "user", "content": "What does this sequence encode?\n\nATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTT"}],
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(response.choices[0].message.content)vLLM applies the same sampling defaults from generation_config.json automatically, unless you override them in the request body. This setup is also what you’d wrap in a function if you want a general-purpose coding or research agent to call Carbon-X specifically for DNA-related sub-questions, passing it named sequences and a question rather than routing genomic data through a general-purpose model.
What’s going on with the tokenizer, and where do people get tripped up?
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
Carbon-X doesn’t use a separate DNA encoder. Instead, it extends a standard BPE vocabulary (inherited from its base model, Qwen3-14B) with 5,515 additional DNA-specific tokens: 4,096 non-overlapping 6-mers, 1,364 “tail” tokens for sequences whose length isn’t a clean multiple of six, and three special markers (<dna>, </dna>, <oov>). DNA and English text live in the same token stream, processed by the same decoder.
A few quirks matter in practice:
- DNA detection is automatic. Any run of at least 6 consecutive A/C/G/T bases gets wrapped in
<dna>...</dna>tags and tokenized as 6-mers. Runs of 12+ bases with up to 20% IUPAC ambiguity codes (N, R, Y, etc.) also get tagged. You can disable this per call withauto_dna_tags=Falseif you need manual control. - Only canonical bases get full resolution. A 6-mer containing anything other than A, C, G, or T collapses into a single
<oov>token, which throws away information. If your sequences contain ambiguity codes and you care about precision, filter to canonical bases before tokenizing. - Odd-length sequences are still handled losslessly. When a sequence’s length isn’t divisible by six, the last 1 to 5 bases get dedicated tail tokens, and decoding reconstructs the original sequence exactly.
- Lowercase and spaced-out sequences get normalized. The tokenizer uppercases lowercase bases and joins sequences that are displayed in space-separated blocks before tagging them.
For prompting, the model card recommends keeping the sequence and the question in the same user message, including any metadata you already have (gene name, strand, coordinates, annotations), and for variant questions, supplying both reference and alternate sequences, or a reference plus an explicit position and base change.
Is Carbon-X worth running locally instead of using a frontier model API?
For DNA-specific tasks, the benchmark numbers suggest yes, especially at scale. Carbon-X averages 66.7% across five DNA benchmarks (BioReason Coding, BioReason Non-SNV, GenomeQA, NextBase, and FrontierVEP) with thinking disabled. That’s 27 points above Qwen3-14B, the base model it was built from, and it leads every other model tested on coding-variant effects and next-base prediction specifically. It sits close to GPT-6.1 Sol’s 67.5% average, and actually loses to larger frontier models on some individual benchmarks like FrontierVEP and BioReason Non-SNV.
Where Carbon-X pulls ahead decisively is cost. Because it answers directly without extended chain-of-thought reasoning, each query consumes far fewer output tokens than a frontier reasoning model working through the same problem. The model card estimates that analyzing the 2.4 million variants of uncertain significance currently in ClinVar would cost about $300 with Carbon-X, versus $32,000 to $4 million using the frontier models evaluated. For any lab running high-volume variant screening, that cost gap is the real argument for local deployment, on top of the data-privacy benefit of never sending sequences to an external API.
It’s also still a competent general assistant: it scores 79.3% prompt-level strict accuracy on IFEval (84.0% averaged across IFEval’s four metrics), so it doesn’t fall apart on ordinary instruction-following tasks when a conversation shifts away from DNA.
Frequently Asked Questions
How much GPU memory does Carbon-X need?
The weights fit on a single 80GB GPU in bfloat16 precision, per the official model card. No officially supported quantized version is documented, so smaller GPUs would require unsupported quantization with some expected accuracy tradeoff.
Does Carbon-X need special software beyond standard Transformers?
Remy doesn't build the plumbing. It inherits it.
Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something.
Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want.
Yes. You need Transformers v5 or later, and you must pass trust_remote_code=True when loading the tokenizer, since the DNA 6-mer tokenization logic is custom code bundled with the model, not part of core Transformers.
Should I leave “thinking” enabled when running Carbon-X?
No. Carbon-X was post-trained and evaluated with enable_thinking=False, and the model card explicitly recommends keeping it that way. All published benchmark scores use this setting.
Can Carbon-X handle DNA sequences with ambiguous bases like N?
Only partially. The tokenizer tags runs of at least 12 bases with up to 20% IUPAC ambiguity codes as DNA, but any 6-mer containing a non-ACGT character collapses into a single <oov> token, losing resolution. Filter to canonical A/C/G/T bases first if precision matters.
How does Carbon-X compare on cost to frontier models for large-scale variant analysis?
The model card estimates that screening all 2.4 million variants of uncertain significance in ClinVar would cost about $300 with Carbon-X versus $32,000 to $4 million with the frontier models evaluated, largely because Carbon-X answers directly without extended reasoning tokens.
