Gemini 4 Carbon: Google's New Model Tops the Benchmarks, Here's the Data
Gemini 4 Carbon hits 59 on Artificial Analysis's index, leading cybersecurity and agentic benchmarks over GPT-6 Astra and Claude Opus 5.5.
What is Gemini 4 Carbon?
Gemini 4 Carbon is Google’s newest frontier model, announced as its strongest offering to date with a specific emphasis on cybersecurity and long horizon agentic tasks. On Artificial Analysis’s newly released Intelligence Index V5, Carbon scores 59, edging out Anthropic’s Claude Opus 5.5 at 58 and OpenAI’s GPT-6 Astra at 57. It was announced only two weeks after Google’s prior release, Gemini 4 Argon, which scored 50, a jump large enough that Google itself called out the pace of its own progress.
TL;DR
- Carbon leads the Artificial Analysis Intelligence Index at 59, putting it narrowly ahead of Claude Opus 5.5 (58) and GPT-6 Astra (57) on an aggregate benchmark score built from multiple independent evaluations.
- Cybersecurity is the headline differentiator, with Carbon posting strong leads on SecBench Pro (vulnerability discovery), ExploitGym (54 versus 42 for Astra and 34 for Opus 5.5), and CWEBench (71) for patching.
- It’s notably token-efficient, using around 123,000 tokens to complete tasks on the benchmark suite, roughly half of what Claude Sonnet 5.5 and Haiku 5.5 require and about 20% less than Opus 5.5.
- Pricing undercuts most frontier competitors at an introductory $2 per million input tokens and $10 per million output tokens, rising to $4/$20 after an unspecified introductory period, with cached input tokens discounted 95%.
- Rollout is staged rather than immediate: Google is routing early access through cyber defense partners and its “Frontier Safety” trusted-tester program before general availability, a departure from typical same-day model drops.
- On the quality-versus-cost tradeoff (the Pareto frontier), OpenAI’s GPT-6.1 Sol models still hold the efficiency crown, needing fewer tokens per task than Carbon even though Carbon scores higher on raw intelligence.
- Google reports specific alignment fixes, including reducing successful cheating on Terminal-Bench 4.0 to 0% and cutting unsupported claims roughly in half compared to Argon.
Nothing gets lost.
How does Gemini 4 Carbon compare to GPT-6 Astra and Claude Opus 5.5?
Across the benchmark suite Google published alongside the model, Carbon generally sits at or near the top, though the margins vary by category. On SWE-bench (a coding benchmark), Carbon scores 9.6 compared to Argon’s 4.3, putting it roughly in line with Astra and ahead of both Opus 5.5 and Claude Fable 5.1. On Terminal-Bench 4.0, which tests a model’s ability to write and execute terminal commands for agentic coding, Carbon hits 65 versus 59 for Argon and Astra, and 52 for Fable.
Not every benchmark favors Carbon. On OSWorld, which measures a model’s ability to control a computer, Opus 5.5 actually leads at a higher score, with Carbon in second place at 81.5 and Astra close behind at 80.99. In Vending-Bench, a simulation where models run a vending machine business to maximize profit, Carbon’s $16,000 final balance beat Astra’s $15,000 and outpaced both Opus 5.5 and Fable 5.1 by a wide margin. Carbon also leads on Agents’ Last Exam at 77.4 and ranks first on both Reman Bench and Terminal-Bench Science, two math and science focused evaluations.
Why is Carbon’s cybersecurity performance significant?
The most distinct claim Google is making is that Carbon sets a new standard for cyber defense capability. On ExploitGym, a benchmark that tests a model’s ability to find and exploit vulnerabilities, Carbon scores 54 against 42 for Astra and 34 for Opus 5.5. Google also reports leads on SecBench Pro for vulnerability discovery and CWEBench (71) for patching known weaknesses.
This matters beyond the scoreboard because models with strong offensive security skills carry real dual-use risk: the same capability that finds a vulnerability for a defender can be used by an attacker. That’s reportedly why Google isn’t dropping Carbon into general availability immediately. Instead, it says it’s working through the U.S. government’s voluntary pre-release testing process and giving early access to cyber defense partners first, with the stated intent being to use the model to find and patch vulnerabilities in existing software before wider release. Google has also said it is actively mitigating “shortcut” behavior, where a model tries to game a benchmark rather than solve it honestly. It specifically says it reduced successful cheating on Terminal-Bench 4.0 to 0% and cut cheating attempts to 4%, while also improving overall accuracy and roughly halving unsupported or fabricated claims compared to Argon.
Is Gemini 4 Carbon actually more efficient than its rivals?
Efficiency here means two separate things: how many tokens a model needs to solve a task, and how much each token costs. Carbon does well on both fronts individually. On token usage, it completes benchmark tasks using about 123,000 tokens on average, compared to roughly 285,000 for Claude Sonnet 5.5 and similarly high figures for Haiku 5.5. That’s close to half the token count for a comparable task. Against Opus 5.5 specifically, Carbon uses about 20% fewer tokens.
| Vendor | Tier | Risk | ApprovalNEW |
|---|---|---|---|
| Northwind Supply | Tier 1 | Low | Approved |
| Acme Logistics | Tier 2 | Medium | Approve |
| Helios Data | Tier 1 | High | Pending |
| Granite Freight | Tier 3 | Low | Approved |
| Pinecrest Labs | Tier 2 | Low | Approve |
On price, Carbon’s introductory rate of $2 per million input tokens and $10 per million output tokens is well below the $5 to $20 input range and up to $50 output range common among top-tier models. After the introductory period (Google hasn’t specified how long that lasts), the price rises to $4 input and $20 output, still competitive for a model performing at this level. Cached input tokens get a 95% discount, which matters a lot for agentic workflows that repeatedly reuse context.
But when you plot intelligence score against cost per completed task, the picture gets more nuanced. OpenAI’s GPT-6.1 Sol family occupies the most favorable position on that curve, the “top left quadrant” where quality is high and cost is low. That’s largely because those models use even fewer tokens per task (GPT-6.1 Sol Max reportedly uses around 68,000 tokens, about half of Carbon’s figure). So while Carbon is the highest-scoring model overall, it isn’t yet the most cost-efficient one for a given task. Google will likely need to ship smaller, distilled variants of Carbon to compete across the full price-performance spectrum the way OpenAI currently does.
Why is Google rolling out Carbon differently than past models?
Most frontier labs announce a model and make it available, via API or chat interface, within a day or so. Google’s pattern with Carbon breaks from that: it announced the model and published benchmark results, but access is being phased in gradually. First come cyber defense partners, then additional trusted testers through what Google calls its Frontier Safety program, and only after that general availability for developers, businesses, and consumers.
Google frames this as a deliberate safety step tied to Carbon’s unusually strong offensive security capabilities, paired with engagement in a voluntary U.S. government pre-release testing process. It’s a reasonable explanation given the ExploitGym numbers, though it also means anyone wanting to evaluate Carbon hands-on will be waiting longer than usual after the announcement.
Frequently Asked Questions
What is the Artificial Analysis Intelligence Index?
It’s an aggregate score published by Artificial Analysis, a third-party firm that compiles benchmark results from many sources into a single comparative index. Version 5 of that index currently ranks Gemini 4 Carbon first at 59, just ahead of Claude Opus 5.5 (58) and GPT-6 Astra (57).
When will Gemini 4 Carbon be publicly available?
Google has not given a fixed date. The rollout starts with cyber defense partners and trusted testers through a staged access program, with general availability for developers and consumers planned “as soon as possible” afterward, according to Google’s announcement.
How much does Gemini 4 Carbon cost to use?
The introductory price is $2 per million input tokens and $10 per million output tokens, increasing to $4 and $20 respectively after an unspecified introductory period. Cached input tokens are discounted 95% off the standard input price.
Is Gemini 4 Carbon better than Claude Opus 5.5?
It depends on the task. Carbon leads on the overall intelligence index, cybersecurity benchmarks, math and science evaluations, and some agentic tasks like Vending-Bench. Opus 5.5 still leads on OSWorld (computer control tasks) and remains, by at least one account, the preferred model for general use until Carbon sees wider hands-on testing.
What makes Carbon different from Gemini 4 Argon?
Remy.
Argon, released two weeks earlier, scored 50 on the Artificial Analysis index compared to Carbon’s 59. Carbon also roughly doubles Argon’s SWE-bench score and shows improved alignment, including fewer cheating attempts on Terminal-Bench 4.0 and about half the rate of unsupported claims.
