How to Run Carbon-X Locally with vLLM for DNA Analysis
Self-host Carbon-X, a 14.8B DNA-language model, on one 80GB GPU with vLLM, and wire it into an agent as a callable tool for DNA tasks.

What is Carbon-X and why run it locally?
Carbon-X is a 14.8B-parameter open-weight language model that reads and writes DNA sequences inside ordinary chat conversations. It comes from the Carbon family released by Hugging Face Bio, built on top of Qwen3-14B and trained jointly on DNA and natural-language text. Because the weights fit on a single 80GB GPU in bfloat16, researchers can run it entirely on their own infrastructure instead of sending genomic data to a third-party API, which matters when the sequences involved are unpublished or clinically sensitive.
TL;DR
- Carbon-X is a 14.8B decoder-only model trained on DNA and text together, with no separate DNA encoder, so it answers questions about raw sequences directly in a chat interface.
- It uses a hybrid tokenizer: standard BPE for English plus 5,515 DNA-specific tokens built from non-overlapping 6-mers, so each DNA token represents six base pairs.
- One 80GB GPU is enough to serve the full model in bfloat16, which is the headline practical detail for anyone deciding whether to self-host versus call a frontier API.
- vLLM serving is a one-line command (
vllm serve HuggingFaceBio/Carbon-X --trust-remote-code), after which the model exposes an OpenAI-compatible chat endpoint on localhost. - Benchmark results show it beats its Qwen3-14B base by 27 points on average across five DNA tasks and comes within a point of GPT-6.1 Sol while costing roughly 1/1,800th as much per problem.
- It can be wrapped as an agent tool that takes a question plus named DNA sequences and routes DNA-specific queries to the locally hosted model instead of a general-purpose LLM.
- Thinking mode should stay disabled: Carbon-X was post-trained and evaluated with
enable_thinking=False, and the model card recommends keeping that setting for both quality and cost reasons.
Remy.
How do you set up Carbon-X with vLLM?
Serving Carbon-X locally requires vLLM and a GPU with enough memory to hold a 14.8B-parameter model in bfloat16, which in practice means a single 80GB card (an H100 or H200-class GPU). The setup has three steps:
- Install vLLM:
pip install -U vllm - Start the server:
vllm serve HuggingFaceBio/Carbon-X --trust-remote-code - Query it through the OpenAI-compatible client once the server is up on
localhost:8000
The --trust-remote-code flag is required because Carbon-X ships a custom tokenizer that handles the DNA-to-token conversion, and that tokenizer logic isn’t part of the standard Transformers library. The same flag is needed if you load the tokenizer directly with AutoTokenizer.from_pretrained() instead of going through vLLM.
Once the server is running, a basic request looks like any other OpenAI chat completion call, except you pass enable_thinking: False inside extra_body.chat_template_kwargs to match how the model was trained and evaluated:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="HuggingFaceBio/Carbon-X",
messages=[{"role": "user", "content": "What does this sequence encode?\n\nATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTT"}],
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(response.choices[0].message.content)
vLLM applies the sampling defaults baked into the model’s generation_config.json (temperature 0.6, top-p 0.95, top-k 20) unless you override them, and those are the exact settings used to produce the benchmark numbers on the model card.
How does the DNA tokenizer actually work?
Carbon-X doesn’t treat DNA as plain text characters. Its tokenizer detects runs of at least 6 adenine/cytosine/guanine/thymine bases and wraps them automatically in <dna>...</dna> tags, then encodes them as non-overlapping 6-mers, meaning each token stands for six base pairs rather than one character. Sequences with up to 20% IUPAC ambiguity codes (like N, R, or Y) are also tagged if they run at least 12 bases long. Lowercase letters get uppercased automatically, and sequences split across space-separated blocks get rejoined before tokenization.
A few practical details matter for anyone feeding real sequences into the model:
- Only A, C, G, and T combine into clean 6-mers. Any 6-mer containing a non-canonical character collapses into a single
<oov>(out-of-vocabulary) token, so filtering to canonical bases first preserves resolution. - Sequences whose length isn’t divisible by 6 don’t lose information: the trailing 1 to 5 bases get dedicated tail tokens, and decoding reconstructs the original sequence exactly.
- DNA that the model generates in its own responses comes wrapped in the same
<dna>tags, which disappear automatically if you decode withskip_special_tokens=True. - Automatic detection can be turned off per call by passing
auto_dna_tags=Falseto the tokenizer or toapply_chat_template(), useful if you’re deliberately sending DNA-like strings that shouldn’t be tagged.
The vocabulary totals 157,184 tokens: 151,669 inherited from Qwen3’s BPE vocabulary for English, plus 5,515 new DNA tokens (4,096 possible 6-mers, 1,364 tail tokens, and three special markers).
How good is Carbon-X compared to frontier models?
Across five DNA benchmarks covering variant-effect prediction, sequence classification, and next-base prediction, Carbon-X averages 66.7% accuracy with thinking disabled. That’s 27 percentage points above Qwen3-14B, the base model it was built from, and it edges out every other open model tested. It lands close to GPT-6.1 Sol’s 67.5% average, and it actually leads the field on two specific tasks: coding-variant effect prediction (BioReason Coding, 74.9%) and next-base prediction (NextBase, 39.4%).
The cost gap is the more striking number. Because Carbon-X answers directly without extended chain-of-thought reasoning, each query uses relatively few output tokens. The model card estimates that running all 2.4 million variants of uncertain significance in ClinVar through Carbon-X would cost around $300, compared to $32,000 to $4 million using the frontier API models evaluated for comparison. That’s a difference of roughly three orders of magnitude, driven by Carbon-X’s small output footprint and the ability to run it on owned hardware rather than pay per-token API pricing.
Carbon-X isn’t uniformly best, though. On complex variant effects (BioReason Non-SNV) and single-base variant effects (FrontierVEP), larger models like Opus 5.5 and GPT-6.1 Sol score higher. The practical takeaway is that Carbon-X trades a few points of peak accuracy on some tasks for a massive cost and privacy advantage, which matters most for high-volume or sensitive analysis.
How do you wire Carbon-X into an agent as a tool?
The model card describes using Carbon-X as a callable tool inside a general-purpose agent, so that DNA-specific questions get routed to the specialized model while everything else goes to a standard LLM. The pattern is straightforward: serve Carbon-X with vLLM as described above, then wrap it in a function that accepts a natural-language question plus a dictionary of named DNA sequences.
from openai import OpenAI
carbon = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
def ask_carbon(question: str, sequences: dict[str, str], context: str = "") -> str:
"""Ask Carbon-X a question about one or more named DNA sequences."""
parts = [context, question] if context else [question]
# sequences get formatted into the prompt alongside the question
...
An agent framework can then expose ask_carbon as a tool definition, so that when a user asks something like “is this variant pathogenic” and supplies a reference and alternate sequence, the orchestrating model calls Carbon-X instead of attempting genomic reasoning itself. This keeps the general-purpose model focused on conversation and task routing while a purpose-trained, cheaper model handles the actual DNA interpretation. For any such integration, the model card’s prompting guidance applies: put the sequence and question in the same message, include known context like gene name or strand, and for variant questions supply both reference and alternate sequences (or a reference plus position and base change) rather than a bare sequence alone.
Is self-hosting Carbon-X worth it?
For teams running DNA analysis at any real volume, or anyone working with sequences that can’t leave internal infrastructure, self-hosting looks like the clear use case the model was built for. The combination of single-GPU deployment, an OpenAI-compatible serving layer via vLLM, and benchmark performance that’s competitive with frontier closed models makes it practical to deploy without a large infrastructure team. The economics reinforce this: at an estimated $5 per H200-hour, the per-problem cost advantage over API-based frontier models is large enough that high-volume workloads (like scanning ClinVar’s full set of uncertain-significance variants) shift from impractical to routine.
The tradeoffs are real too. Carbon-X slightly trails the best frontier models on some benchmark categories, its context window tops out at 40,960 tokens, and it still requires datacenter-class GPU hardware (an 80GB card isn’t consumer hardware). For occasional, low-volume queries, calling a hosted frontier model may remain simpler. But for sustained genomic analysis work, particularly anything involving sensitive or unpublished sequences, local deployment removes both the data-exposure risk and the per-token cost that scales badly with large variant datasets.
Frequently Asked Questions
What GPU do I need to run Carbon-X locally?
A single 80GB GPU (such as an H100 or H200) is sufficient to serve the full 14.8B-parameter model in bfloat16 precision, according to the model card.
Does Carbon-X require any special installation steps beyond vLLM?
Yes. Because Carbon-X uses a custom DNA tokenizer, you need to pass trust_remote_code=True when loading the tokenizer directly, or --trust-remote-code when starting the vLLM server.
Should I enable “thinking” mode when running Carbon-X?
No. Carbon-X was post-trained and evaluated with enable_thinking=False, and the model card recommends keeping thinking disabled for both accuracy and lower output-token cost.
How does Carbon-X represent DNA differently from plain text?
It tokenizes DNA as non-overlapping 6-mers (one token per six base pairs) using a dedicated vocabulary of 5,515 DNA tokens, rather than treating DNA as ordinary character-level text, and it automatically detects and tags DNA runs of six or more canonical bases in any prompt.
Can Carbon-X replace a general-purpose LLM entirely?
No. It’s designed to be called as a specialized tool for DNA-specific questions within a broader agent setup, handling tasks like variant-effect prediction and sequence classification, while a general-purpose model handles everything else.



