How to Run Carbon-X Locally with vLLM for DNA Analysis
Self-host Carbon-X, a 14.8B DNA-language model, on one 80GB GPU with vLLM, and wire it into an agent as a callable tool for DNA tasks.
What is Carbon-X and why run it locally?
Carbon-X is a 14.8B-parameter open-weight language model that reads and writes DNA sequences inside ordinary chat conversations. It comes from the Carbon family released by Hugging Face Bio, built on top of Qwen3-14B and trained jointly on DNA and natural-language text. Because the weights fit on a single 80GB GPU in bfloat16, researchers can run it entirely on their own infrastructure instead of sending genomic data to a third-party API, which matters when the sequences involved are unpublished or clinically sensitive.
TL;DR
- Carbon-X is a 14.8B decoder-only model trained on DNA and text together, with no separate DNA encoder, so it answers questions about raw sequences directly in a chat interface.
- It uses a hybrid tokenizer: standard BPE for English plus 5,515 DNA-specific tokens built from non-overlapping 6-mers, so each DNA token represents six base pairs.
- One 80GB GPU is enough to serve the full model in bfloat16, which is the headline practical detail for anyone deciding whether to self-host versus call a frontier API.
- vLLM serving is a one-line command (
vllm serve HuggingFaceBio/Carbon-X --trust-remote-code), after which the model exposes an OpenAI-compatible chat endpoint on localhost. - Benchmark results show it beats its Qwen3-14B base by 27 points on average across five DNA tasks and comes within a point of GPT-6.1 Sol while costing roughly 1/1,800th as much per problem.
- It can be wrapped as an agent tool that takes a question plus named DNA sequences and routes DNA-specific queries to the locally hosted model instead of a general-purpose LLM.
- Thinking mode should stay disabled: Carbon-X was post-trained and evaluated with
enable_thinking=False, and the model card recommends keeping that setting for both quality and cost reasons.
Remy doesn't write the code. It manages the agents who do.
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
How do you set up Carbon-X with vLLM?
Serving Carbon-X locally requires vLLM and a GPU with enough memory to hold a 14.8B-parameter model in bfloat16, which in practice means a single 80GB card (an H100 or H200-class GPU). The setup has three steps:
- Install vLLM:
pip install -U vllm - Start the server:
vllm serve HuggingFaceBio/Carbon-X --trust-remote-code - Query it through the OpenAI-compatible client once the server is up on
localhost:8000
The --trust-remote-code flag is required because Carbon-X ships a custom tokenizer that handles the DNA-to-token conversion, and that tokenizer logic isn’t part of the standard Transformers library. The same flag is needed if you load the tokenizer directly with AutoTokenizer.from_pretrained() instead of going through vLLM.
Once the server is running, a basic request looks like any other OpenAI chat completion call, except you pass enable_thinking: False inside extra_body.chat_template_kwargs to match how the model was trained and evaluated:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="HuggingFaceBio/Carbon-X",
messages=[{"role": "user", "content": "What does this sequence encode?\n\nATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTT"}],
extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(response.choices[0].message.content)vLLM applies the sampling defaults baked into the model’s generation_config.json (temperature 0.6, top-p 0.95, top-k 20) unless you override them, and those are the exact settings used to produce the benchmark numbers on the model card.
How does the DNA tokenizer actually work?
Carbon-X doesn’t treat DNA as plain text characters. Its tokenizer detects runs of at least 6 adenine/cytosine/guanine/thymine bases and wraps them automatically in <dna>...</dna> tags, then encodes them as non-overlapping 6-mers, meaning each token stands for six base pairs rather than one character. Sequences with up to 20% IUPAC ambiguity codes (like N, R, or Y) are also tagged if they run at least 12 bases long. Lowercase letters get uppercased automatically, and sequences split across space-separated blocks get rejoined before tokenization.
A few practical details matter for anyone feeding real sequences into the model:
- Only A, C, G, and T combine into clean 6-mers. Any 6-mer containing a non-canonical character collapses into a single
<oov>(out-of-vocabulary) token, so filtering to canonical bases first preserves resolution. - Sequences whose length isn’t divisible by 6 don’t lose information: the trailing 1 to 5 bases get dedicated tail tokens, and decoding reconstructs the original sequence exactly.
- DNA that the model generates in its own responses comes wrapped in the same
<dna>tags, which disappear automatically if you decode withskip_special_tokens=True. - Automatic detection can be turned off per call by passing
auto_dna_tags=Falseto the tokenizer or toapply_chat_template(), useful if you’re deliberately sending DNA-like strings that shouldn’t be tagged.
The vocabulary totals 157,184 tokens: 151,669 inherited from Qwen3’s BPE vocabulary for English, plus 5,515 new DNA tokens (4,096 possible 6-mers, 1,364 tail tokens, and three special markers).
How good is Carbon-X compared to frontier models?
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
Across five DNA benchmarks covering variant-effect prediction, sequence classification, and next-base prediction, Carbon-X averages 66.7% accuracy with thinking disabled. That’s 27 percentage points above Qwen3-14B, the base model it was built from, and it edges out every other open model tested. It lands close to GPT-6.1 Sol’s 67.5% average, and it actually leads the field on two specific tasks: coding-variant effect prediction (BioReason Coding, 74.9%) and next-base prediction (NextBase, 39.4%).
The cost gap is the more striking number. Because Carbon-X answers directly without extended chain-of-thought reasoning, each query uses relatively few output tokens. The model card estimates that running all 2.4 million variants of uncertain significance in ClinVar through Carbon-X would cost around $300, compared to $32,000 to $4 million using the frontier API models evaluated for comparison. That’s a difference of roughly three orders of magnitude, driven by Carbon-X’s small output footprint and the ability to run it on owned hardware rather than pay per-token API pricing.
Carbon-X isn’t uniformly best, though. On complex variant effects (BioReason Non-SNV) and single-base variant effects (FrontierVEP), larger models like Opus 5.5 and GPT-6.1 Sol score higher. The practical takeaway is that Carbon-X trades a few points of peak accuracy on some tasks for a massive cost and privacy advantage, which matters most for high-volume or sensitive analysis.
How do you wire Carbon-X into an agent as a tool?
The model card describes using Carbon-X as a callable tool inside a general-purpose agent, so that DNA-specific questions get routed to the specialized model while everything else goes to a standard LLM. The pattern is straightforward: serve Carbon-X with vLLM as described above, then wrap it in a function that accepts a natural-language question plus a dictionary of named DNA sequences.
from openai import OpenAI
carbon = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
def ask_carbon(question: str, sequences: dict[str, str], context: str = "") -> str:
"""Ask Carbon-X a question about one or more named DNA sequences."""
parts = [context, question] if context else [question]
# sequences get formatted into the prompt alongside the question
...An agent framework can then expose ask_carbon as a tool definition, so that when a user asks something like “is this variant pathogenic” and supplies a reference and alternate sequence, the orchestrating model calls Carbon-X instead of attempting genomic reasoning itself. This keeps the general-purpose model focused on conversation and task routing while a purpose-trained, cheaper model handles the actual DNA interpretation. For any such integration, the model card’s prompting guidance applies: put the sequence and question in the same message, include known context like gene name or strand, and for variant questions supply both reference and alternate sequences (or a reference plus position and base change) rather than a bare sequence alone.
Is self-hosting Carbon-X worth it?
For teams running DNA analysis at any real volume, or anyone working with sequences that can’t leave internal infrastructure, self-hosting looks like the clear use case the model was built for. The combination of single-GPU deployment, an OpenAI-compatible serving layer via vLLM, and benchmark performance that’s competitive with frontier closed models makes it practical to deploy without a large infrastructure team. The economics reinforce this: at an estimated $5 per H200-hour, the per-problem cost advantage over API-based frontier models is large enough that high-volume workloads (like scanning ClinVar’s full set of uncertain-significance variants) shift from impractical to routine.
Other agents start typing. Remy starts asking.
Scoping, trade-offs, edge cases — the real work. Before a line of code.
The tradeoffs are real too. Carbon-X slightly trails the best frontier models on some benchmark categories, its context window tops out at 40,960 tokens, and it still requires datacenter-class GPU hardware (an 80GB card isn’t consumer hardware). For occasional, low-volume queries, calling a hosted frontier model may remain simpler. But for sustained genomic analysis work, particularly anything involving sensitive or unpublished sequences, local deployment removes both the data-exposure risk and the per-token cost that scales badly with large variant datasets.
Frequently Asked Questions
What GPU do I need to run Carbon-X locally?
A single 80GB GPU (such as an H100 or H200) is sufficient to serve the full 14.8B-parameter model in bfloat16 precision, according to the model card.
Does Carbon-X require any special installation steps beyond vLLM?
Yes. Because Carbon-X uses a custom DNA tokenizer, you need to pass trust_remote_code=True when loading the tokenizer directly, or --trust-remote-code when starting the vLLM server.
Should I enable “thinking” mode when running Carbon-X?
No. Carbon-X was post-trained and evaluated with enable_thinking=False, and the model card recommends keeping thinking disabled for both accuracy and lower output-token cost.
How does Carbon-X represent DNA differently from plain text?
It tokenizes DNA as non-overlapping 6-mers (one token per six base pairs) using a dedicated vocabulary of 5,515 DNA tokens, rather than treating DNA as ordinary character-level text, and it automatically detects and tags DNA runs of six or more canonical bases in any prompt.
Can Carbon-X replace a general-purpose LLM entirely?
No. It’s designed to be called as a specialized tool for DNA-specific questions within a broader agent setup, handling tasks like variant-effect prediction and sequence classification, while a general-purpose model handles everything else.
