Carbon-X Benchmarks: How a 14B DNA Model Compares to GPT-6.1
Carbon-X averages 66.7% across five DNA benchmarks, close to GPT-6.1 Sol, at roughly 1/1,800 the inference cost of frontier models.
What is Carbon-X and why does its benchmark score matter?
Carbon-X is a 14.8-billion-parameter open-weight language model that reads and writes DNA sequences inside ordinary chat conversations. It predicts whether a mutation is benign or pathogenic, flags regulatory regions, classifies organisms from sequence data, and predicts upcoming bases, all in plain English output. Across five DNA benchmarks it averages 66.7%, within a point of GPT-6.1 Sol (67.5%), while running on a single 80GB GPU instead of a frontier API. That combination of near-frontier accuracy and low cost is the headline result, and it’s why a model built by extending Qwen3-14B is drawing attention from people who work on variant interpretation at scale.
TL;DR
- Carbon-X averages 66.7% across five DNA benchmarks (BioReason Coding, BioReason Non-SNV, GenomeQA, NextBase, FrontierVEP) with thinking disabled, landing just under GPT-6.1 Sol’s 67.5% and ahead of Opus 5.5, Gemini 4 Argon, GLM-5.3, Kimi K3, and DeepSeek V4.1 Flash on average.
- It wins outright on two tasks: coding-region variant effects (74.9% vs GPT-6.1 Sol’s 71.0%) and next-base prediction (39.4%, well clear of every other model tested, including Opus 5.5’s near-chance 1.1%).
- The cost gap is the real story: Carbon-X answers directly without extended reasoning, so each problem costs a fraction of a frontier model’s output-token bill, an estimated 1/1,800th per problem based on measured throughput on an H200 GPU at $5/hour versus standard API pricing.
- A real-world estimate: scoring all 2.4 million variants of uncertain significance in ClinVar would cost about $300 with Carbon-X, versus $32,000 to $4 million using the frontier models evaluated.
- It’s a single decoder, not a pipeline: DNA and text share one token stream, with a hybrid tokenizer that treats DNA as non-overlapping 6-mers (one token = 6 base pairs) and text as standard BPE.
- It stays a usable assistant: Carbon-X scores 79.3% on IFEval prompt-level strict accuracy, meaning the DNA specialization didn’t gut its ability to follow instructions.
- Weights are open and self-hostable: the full 14.8B model fits on one 80GB GPU in bfloat16, which matters for labs that can’t send unpublished sequences to a third-party API.
Remy doesn't build the plumbing. It inherits it.
Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something.
Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want.
How does Carbon-X actually process DNA?
Most genomics models bolt a dedicated DNA encoder onto a language model, or run two separate pipelines that get stitched together downstream. Carbon-X does neither. It’s a single decoder-only transformer (40 layers, hidden size 5120, grouped-query attention with 40 query heads and 8 KV heads, SwiGLU activations) that treats DNA and English as one continuous token stream.
The trick is the tokenizer. Carbon-X keeps the existing Qwen3 BPE vocabulary for English text and adds 5,515 new tokens for DNA: 4,096 of them cover every possible non-overlapping 6-mer (a run of 6 bases), plus tail tokens for sequences whose length isn’t a clean multiple of six, plus special markers that wrap DNA spans in the token stream. When you paste a raw sequence into a prompt, the tokenizer automatically detects runs of at least 6 canonical bases and tags them, so you don’t need to pre-format anything.
This matters for two practical reasons. First, because DNA tokens live in the same embedding space as English tokens, the model can reason about a sequence and explain that reasoning in natural language within a single forward pass, no handoff between a specialist encoder and a separate language model. Second, it’s one of the reasons inference is cheap: a 6-mer tokenization means one token represents six base pairs, so even long sequences compress into a manageable context (the model supports up to 40,960 tokens).
How was Carbon-X trained to get these scores?
Carbon-X starts from Qwen3-14B, a general-purpose open language model, not from scratch. The DNA vocabulary is grafted on with randomly initialized embeddings, and the first training phase freezes the rest of the model so only those new embeddings learn, before unfreezing everything.
From there, the model goes through 302 billion tokens of mid-training on a deliberately mixed diet: 55% raw DNA (a mix of eukaryotic genomes, mRNA, and prokaryotic genomes), 20% general English text, 15% biology-specific text (papers, forum posts, annotated sequence descriptions), and 10% instruction-following data. The team notes that the learning rate was the single most sensitive setting at this stage: too low and the model never picks up DNA, too high and it forgets English entirely.
After mid-training comes supervised fine-tuning on curated instruction data (variant-effect reasoning, sequence classification, DNA editing operations, next-base prediction, biology Q&A), followed by two rounds of reinforcement learning using GRPO with verifiable, code-checked rewards. The reward functions directly check things like whether a variant call is correct and whether the model’s stated rationale is actually grounded in the sequence, rather than just rewarding a plausible-sounding answer.
How does Carbon-X compare to GPT-6.1 Sol and Opus 5.5 on specific tasks?
The headline average (66.7% vs 67.5% for GPT-6.1 Sol) hides meaningful task-by-task differences, and those differences are arguably more useful than the aggregate number.
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
On BioReason Coding (predicting benign vs. pathogenic mutations in protein-coding regions, with the gene name stripped from the prompt), Carbon-X leads everything tested at 74.9%, ahead of GPT-6.1 Sol (71.0%) and Opus 5.5 (70.3%). On NextBase, a per-base next-token prediction task over DNA, Carbon-X again leads at 39.4%, while Opus 5.5 collapses to near-chance performance (1.1%), suggesting general-purpose frontier models aren’t well calibrated for raw sequence continuation even when they’re strong at variant reasoning.
Carbon-X trails on the two tasks involving more complex variant types and single-nucleotide precision: BioReason Non-SNV (insertions, deletions, and other non-point mutations) goes to Opus 5.5 at 94.8% versus Carbon-X’s 85.1%, and FrontierVEP, a single-nucleotide variant benchmark using reference sequences up to 8,192 bases, also goes to Opus 5.5 (76.2% vs Carbon-X’s 70.4%). On GenomeQA, a multiple-choice benchmark spanning enhancers, splice sites, organism classification, and transcription-factor binding, GPT-6.1 Sol edges ahead at 67.2% to Carbon-X’s 63.6%.
Against open models, the gap is much wider. GLM-5.3, Kimi K3, and DeepSeek V4.1 Flash all land between roughly 40% and 50% average, well below Carbon-X’s 66.7%.
Is Carbon-X worth using over a frontier model?
The answer depends on what you’re optimizing for. If you need the single highest accuracy on a narrow, high-stakes variant call, Opus 5.5 or GPT-6.1 Sol post better numbers on non-SNV and single-nucleotide variant tasks. But for anything involving volume, cost or data sensitivity becomes the deciding factor.
Carbon-X doesn’t use extended reasoning. It answers directly, so each inference call uses a small number of output tokens. That’s the mechanical reason behind the cost estimate: analyzing all 2.4 million variants of uncertain significance currently in ClinVar would run about $300 with Carbon-X, versus a range of $32,000 to $4 million with the frontier closed models evaluated, based on standard API pricing against Carbon-X’s measured throughput on an H200 GPU. For labs doing large-scale variant triage, reanalysis of legacy data, or any workflow that touches millions of sequences, that cost difference changes what’s even feasible to run.
There’s also a data-governance angle. Because the full model fits on a single 80GB GPU in bfloat16, researchers working with unpublished sequences, patient data, or anything under data-sharing restrictions can run Carbon-X entirely on their own infrastructure instead of sending sequences to a third-party API.
What are Carbon-X’s limitations?
Carbon-X is not uniformly better than frontier models. It loses on complex variant types (indels and other non-SNV changes) and on single-nucleotide variant prediction when judged against Opus 5.5 specifically, and it’s slightly behind GPT-6.1 Sol on general DNA understanding questions (GenomeQA). Its tokenizer also only forms full-resolution 6-mers from canonical A/C/G/T bases; sequences containing ambiguity codes like N fall back to out-of-vocabulary tokens unless filtered first. And because it was trained and evaluated with “thinking” disabled, it’s tuned for fast, direct answers rather than long chains of reasoning, which is part of why it’s cheap but also means it may not benefit from reasoning-style prompting the way general LLMs do.
Frequently Asked Questions
What benchmarks was Carbon-X tested on?
Five: BioReason Coding and BioReason Non-SNV (variant-effect prediction from the published BioReason literature), GenomeQA (sequence understanding, from the literature), and two new benchmarks built for this evaluation, FrontierVEP (single-nucleotide variant classification) and NextBase (next-base sequence prediction).
How much cheaper is Carbon-X than GPT-6.1 Sol or Opus 5.5?
Remy doesn't write the code. It manages the agents who do.
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
The model card estimates roughly 1/1,800th the inference cost per problem compared to the frontier models evaluated, based on Carbon-X’s measured throughput on an H200 GPU at $5 per hour versus standard API pricing for the comparison models.
Can Carbon-X run on consumer or single-GPU hardware?
Yes. At 14.8 billion parameters, the full model fits on a single 80GB GPU in bfloat16 precision, and it serves efficiently with vLLM.
Does Carbon-X replace a general-purpose assistant?
Not entirely, but it stays usable as one. It scores 79.3% prompt-level strict accuracy on IFEval, a standard instruction-following benchmark, suggesting the DNA-focused training didn’t erase its general chat and instruction-following ability.
What architecture does Carbon-X use?
It’s a decoder-only transformer built by extending Qwen3-14B, with a hybrid tokenizer that encodes DNA as non-overlapping 6-mers alongside a standard byte-pair encoding vocabulary for text, all processed in a single token stream with no separate DNA encoder.
