How to Run Carbon-X Locally for DNA Variant Analysis
Carbon-X is a 14.8B open DNA language model that fits on one 80GB GPU. Here's how to set it up with vLLM or Transformers.
What is Carbon-X?
Carbon-X is a 14.8-billion-parameter open-weight language model that reads and writes DNA sequences inside ordinary chat conversations. It runs on a single 80GB GPU in bfloat16, serves through vLLM or Hugging Face Transformers, and answers questions about mutation effects, gene-regulatory regions, species classification and next-base prediction without a separate DNA encoder. Because it fits on one GPU, researchers can run it entirely on their own infrastructure instead of sending sequence data to a third-party API.
TL;DR
- Carbon-X is a 14.8B decoder-only model built on Qwen3-14B that was extended with a DNA-specific tokenizer and trained jointly on genomic and natural-language data.
- It fits on a single 80GB GPU in bfloat16, which makes local deployment realistic for labs that don’t want sensitive or unpublished sequences leaving their own hardware.
- It uses a hybrid tokenizer that treats DNA as non-overlapping 6-mers (one token per 6 base pairs) while keeping a standard BPE vocabulary for English text, all in one token stream.
- Serving options include vLLM and Transformers v5, with the vLLM path exposing an OpenAI-compatible API that’s convenient for wiring into agent pipelines.
- Benchmark results show it beating every open model tested and running close to GPT-6.1 Sol’s score at roughly 1/1,800 of the per-problem inference cost.
- It can act as a callable tool for a general-purpose agent, letting a larger assistant delegate DNA-specific questions to Carbon-X while handling everything else itself.
- Thinking mode should stay disabled, since Carbon-X was post-trained and evaluated that way, and enabling extended reasoning isn’t the intended use pattern.
What hardware do you need to run Carbon-X?
Carbon-X’s weights fit on a single 80GB GPU in bfloat16. That’s the headline hardware spec in its model card, and it’s the detail that makes local deployment practical for a biology or bioinformatics team. An 80GB card (the class that includes H100 and H200 GPUs) is still a serious investment, but it’s a world away from needing multi-GPU clusters or model parallelism just to load the weights.
The model’s architecture explains why it stays compact: 40 transformer layers, a hidden size of 5,120, grouped-query attention with 40 query heads and 8 key/value heads, SwiGLU activations, and RMSNorm with QK-norm. Untied embeddings and a vocabulary padded to 157,184 tokens (151,669 BPE tokens plus 5,515 DNA-specific tokens) round out the design. None of that changes the basic resourcing question: one 80GB GPU, bfloat16 precision, and you can load and query the full model.
Carbon-X also answers directly rather than reasoning step by step at inference time, since it was evaluated with thinking disabled. That keeps output token counts low, which matters for both latency and cost when you’re running the model yourself and paying for GPU time rather than per-token API pricing.
How do you set up Carbon-X with Transformers?
The Transformers path is the simplest way to get Carbon-X running for one-off queries or scripted analysis. Two things matter here: you need Transformers v5 or later, and you need trust_remote_code=True when loading the tokenizer, because the DNA tokenizer is custom code shipped with the model.
Install the latest Transformers release:
pip install -U transformersThen load the model and tokenizer:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "HuggingFaceBio/Carbon-X"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda")From there you build a standard chat message, apply the chat template with enable_thinking=False, and call generate(). A typical use case is a variant-effect question: you give the model a reference sequence and a mutated version of it, then ask whether the variant looks benign or pathogenic and why. The model responds in plain English, with any DNA it generates wrapped in <dna>...</dna> tags that get stripped automatically if you decode with skip_special_tokens=True.
Generation uses the sampling defaults baked into generation_config.json (temperature 0.6, top-p 0.95, top-k 20). These are the same settings used to produce every benchmark score reported for the model, so changing them means you’re no longer comparing apples to apples with the published numbers.
How do you serve Carbon-X with vLLM?
For anything beyond quick scripts, vLLM is the more production-ready option. It gives you an OpenAI-compatible HTTP endpoint, which makes Carbon-X easy to plug into existing tooling that already speaks the OpenAI client format.
Install vLLM and start the server:
pip install -U vllm
vllm serve HuggingFaceBio/Carbon-X --trust-remote-codeThen query it like any OpenAI-compatible endpoint:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="HuggingFaceBio/Carbon-X",
messages=[{"role": "user", "content": "What does this sequence encode?\n\nATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTT"}],
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(response.choices[0].message.content)Other agents start typing. Remy starts asking.
Scoping, trade-offs, edge cases — the real work. Before a line of code.
vLLM applies the same sampling defaults from generation_config.json unless you override them in the request body, so behavior stays consistent between the Transformers and vLLM paths.
How does the DNA tokenizer actually work?
Carbon-X doesn’t use a separate encoder for genetic sequences. Instead, it extends a text tokenizer (built on Qwen3’s BPE vocabulary) with 5,515 additional tokens dedicated to DNA: 4,096 non-overlapping 6-mers, 1,364 tail tokens for sequences whose length isn’t a multiple of six, and three special markers (<dna>, </dna>, <oov>).
DNA inside a prompt is detected and tagged automatically. Any run of at least 6 A/C/G/T bases gets wrapped in <dna>...</dna> and split into 6-mers, so a 36-base sequence becomes 6 tokens. Sequences with IUPAC ambiguity codes (things like N, R, Y) are tagged too if they’re at least 12 bases long and no more than 20% ambiguous. Lowercase bases get uppercased automatically, and any non-standard 6-mer, such as one containing an “N”, collapses to a single <oov> token rather than keeping full resolution. If you need exact base-level fidelity, filter sequences down to canonical A/C/G/T first.
This matters practically because it means you can paste raw sequences directly into a chat message alongside plain English, and the tokenizer sorts out what’s DNA and what’s text without manual preprocessing. Automatic detection can be turned off per call by passing auto_dna_tags=False if you want manual control.
Is Carbon-X worth running locally?
For teams working with sequence data they can’t or don’t want to send to an external API, the case is straightforward. Carbon-X’s benchmark average across five DNA tasks (variant effects, sequence understanding, next-base prediction) is 66.7% with thinking disabled, roughly 27 points above Qwen3-14B, the base model it was built from. That puts it ahead of every other open model tested, and within about a point of GPT-6.1 Sol’s 67.5% average, a frontier closed model, while reportedly costing about 1/1,800 as much per problem to run.
The cost gap compounds at scale. Analyzing the 2.4 million variants of uncertain significance currently in ClinVar is estimated at around $300 using Carbon-X, versus $32,000 to $4 million for the frontier models evaluated alongside it. For labs doing large batch variant annotation, that difference changes what’s feasible to run at all.
Carbon-X isn’t uniformly best, though. On BioReason Non-SNV (complex variant effects like insertions and deletions) and FrontierVEP (single-base variant effects), Opus 5.5 scores higher. On GenomeQA, GPT-6.1 Sol edges it out. Carbon-X’s clearest wins are coding-variant effects and next-base prediction, where it leads every model tested. The practical takeaway: it’s a strong generalist for DNA tasks and the best open option by a wide margin, but not a universal upgrade over every frontier closed model on every specific benchmark.
Can Carbon-X work as a tool inside a larger agent?
Everyone else built a construction worker.
We built the contractor.
One file at a time.
UI, API, database, deploy.
Yes, and the model card describes this as an intended use case. Rather than replacing a general-purpose assistant, Carbon-X is meant to be called by one. You serve it with vLLM, then wrap it in a function that accepts a question plus named DNA sequences and forwards them to the Carbon-X endpoint. A broader agent handling varied tasks can route anything involving DNA to this function while keeping everything else, general reasoning, writing, search, on its main model.
This pattern keeps Carbon-X doing what it’s specialized for (sequence-level reasoning) without forcing a general assistant to be fine-tuned on genomics or forcing Carbon-X to handle tasks outside its training, like open-ended conversation unrelated to biology.
Frequently Asked Questions
How much VRAM does Carbon-X need?
It fits on a single 80GB GPU in bfloat16 precision, according to its model card. That covers the full 14.8B-parameter model without quantization or multi-GPU sharding.
Do I need special code to load Carbon-X?
Yes. The tokenizer uses custom code for its DNA-handling logic, so you must pass trust_remote_code=True when loading it, and you need Transformers v5 or later installed.
Should I enable thinking mode for better answers?
No. Carbon-X was post-trained and evaluated with thinking disabled (enable_thinking=False), and that’s the configuration the published benchmark scores reflect. The model card recommends keeping it that way.
What context length does Carbon-X support?
It supports a 40,960-token context window, which comes from RoPE positional embeddings with a base theta of 1x10^6.
How does Carbon-X compare to using a frontier API model for DNA tasks?
It scores close to GPT-6.1 Sol on average across five DNA benchmarks (66.7% versus 67.5%) while costing an estimated 1/1,800th as much per problem, and it outperforms other open models tested across the board.
