Claude Opus 5.1 Benchmark Review: Coding Scores and Real Costs
Claude Opus 5.1 tested on coding, 3D, and agentic benchmarks against Opus 5, GLM, Kimi, and Qwen, plus real API cost and cache-write breakdowns.
What is Claude Opus 5.1 and how does it perform on coding benchmarks?
Claude Opus 5.1, released by Anthropic on September 1st, posts the largest coding benchmark gains Anthropic has published for a single point release. On Terminal Bench 2.0, it jumps from 24.7% (Opus 5) to 52.6%, a 27.9 point gain, ahead of GPT-5.1-Codex at 22.4% (Anthropic reports a standard error of 3.5 to 4.5 points on that score). Terminal Bench 4.0 rises from 42% to 55.8%. Independent hands-on testing using a custom coding benchmark (front end, 3JS, SVG, math, and long-horizon agentic tasks) scored it 74 out of 80, or 92.5%, the highest result recorded on that bench across any model tested, including GLM 5.3, Kimi K3, and Qwen 3 Max.
TL;DR
- Opus 5.1 scored 92.5% on an independent coding benchmark, beating GLM 5.3 (91.1%), Opus 5 (82.5%), and Qwen 3 Max (81.2%), making it the first model on that test to break 90%.
- The biggest single jump was on a hard 3D wristwatch task, going from a 4/10 (previous Opus generation) to a 9/10, the best score any model has posted on that question.
- Cache read pricing dropped 75%, from $1 to $0.25 per million tokens, but cache write pricing stayed flat, so real-world bills are now dominated by cache writes rather than reads.
- One published real-world invoice showed cache writes accounting for 65.8% of total cost while representing only 9.4% of total tokens processed.
- Community reports describe a silent fallback to Opus for certain security or low-level systems prompts, meaning users may pay Opus-tier costs without realizing it.
- Three API-breaking changes (no forced tool use, thinking blocks locked to Opus 5.1/Claude Mythos 5.1, and append-only conversations) function as anti-distillation measures against extracting the model’s raw reasoning.
- Anthropic’s own documentation admits the model writes denser, less formatted prose, which some reviewers find harder to read despite fewer generic AI phrasings.
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
How does Opus 5.1 compare to Opus 5 and other models on real coding tasks?
Across an eight-question custom benchmark covering front-end animation, 3JS rendering, SVG generation, a math logic problem, a long-horizon agentic build, and a difficult 3D simulation, Opus 5.1 scored a perfect 10/10 on three tasks: a three-elevator simulation with correct queuing logic, a permutation math problem (answer: 2460), and a long-horizon task that generated a Pandas dataset, fine-tuned Gemma 2B on it, and served a local web UI, all without human intervention.
It scored 9/10 on a 3JS contact lens case (down one point from Opus 5, which is the only model to hit a perfect 10 there), 9/10 on a folding table 3JS animation (GLM 5.3 is the only model with a 10 on this one), 9/10 on an SVG panda-eating-a-burger illustration, and 8/10 on a bow-and-arrow game with a leaderboard (tied with Opus 5, behind Grok 4.5 and Qwen 3.8 Max, which scored 10).
The standout result was the 3D wristwatch task, widely considered the hardest question on the bench: real-time smooth hand movement, date and day display, and two time zones. Opus 5.1 scored 9/10, up from 4/10 on Opus 5, and ahead of the previous best score of 7/10 shared by Deepseek V4 Pro and GLM 5.3.
Final tallies: Opus 5.1 (92.5%), GLM 5.3 (91.1%), Opus 5 (82.5%), Qwen 3.8 Max (81.2%), Opus 4.8 (80%), Opus 5 and Kimi K3 tied at 77.5%.
What does Opus 5.1 actually cost in real usage?
List pricing is unchanged from Opus 5 at $10 per million input tokens and $50 per million output tokens. The only price change is cache reads, cut from $1 to $0.25 per million tokens, a 75% reduction. Cache writes remain at $12.50 per million (5-minute cache) and $20 per million (1-hour cache). Batch API pricing stays at $5/$25.
In one recorded test session totaling $3.60, the token breakdown looked like this: 4,900 input tokens, 22,900 output tokens, and 1.2 million cache read tokens across 18 requests, with 92% of input tokens served from cache. Output tokens cost about $1.15, cache reads about $0.30, and fresh input about $0.05, adding up to roughly $1.50. The remaining $2.10, about 58% of the total bill, came from cache writes, working out to roughly 105,000 tokens written at the 1-hour rate. Running the same session at Opus 5 pricing (where cache reads cost $1 instead of $0.25) would have cost about $4.50, a 20% saving that roughly matches Anthropic’s claimed 25% typical reduction.
A separate, larger real-world invoice reported 573 million total tokens for a bill of $1,023.78. Cache reads made up 89.8% of tokens but only 12.6% of cost ($128.67). Cache writes were just 9.4% of tokens but 65.8% of cost ($673.87). Output tokens were 21.4% of tokens and 21.4% of cost ($219.29). The pattern is consistent across both the small test session and the large invoice: cheaper cache reads shift the cost burden onto cache writes, which didn’t get a price cut.
Is the cache pricing change actually a good deal?
Remy doesn't build the plumbing. It inherits it.
Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something.
Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want.
It depends heavily on workload shape. Anthropic states the changes deliver about 25% savings on typical workloads and up to 45% on highly agentic ones, based on internal August usage where cache reads represented roughly 65% of total billing on long agent loops. For short, one-off calls with no cache prefix, the discount is effectively zero, since there’s nothing being read from cache repeatedly.
The complication is that Opus 5.1 tends to do more whole-file rewrites instead of targeted edits, which increases output tokens and cache write volume for the same task. That behavior partially offsets the cache read discount, since cache writes are unchanged in price and now represent a larger share of typical bills. One team reported roughly 30% real cost reduction against Opus 5 on their evaluation workloads, and found that at 95% cache hit rates, the ideal context window size roughly doubles from 200,000 to 400,000 tokens before costs climb again.
What are the biggest complaints from real-world use?
Four issues recur across community reports and independent testing:
Cache write costs. As shown above, cache writes now dominate billing on agentic workloads because they didn’t receive a price cut while cache reads did. The savings message is real but incomplete without accounting for this shift.
Silent fallback to Opus on security-adjacent prompts. Multiple users report that Opus 5.1 will write unsafe or low-level systems code (such as Win32 Rust) but then refuse to review that same code when flagged as potentially dangerous. Mentioning Linux capabilities (CMP) has reportedly triggered refusals for some users. One estimate circulating in the community suggests roughly 80% of prompts intended for the base model on networking-related tasks get silently routed to Opus by an alignment classifier, meaning some users may be billed differently and get different answers than expected without visibility into the switch.
Anti-distillation restrictions. Three breaking API changes target extraction of the model’s internal reasoning: forced tool use (tool_choice: any or a named tool) now returns a 400 error because it would bypass always-on thinking; thinking blocks can only be read by Opus 5.1 or Claude Mythos 5.1, so switching models mid-conversation silently drops prior reasoning and forces a replan; and for accounts created after August 31st, editing any earlier turn, system prompt, or tools array invalidates all subsequent thinking blocks, making conversations effectively append-only. Some users describe the model as difficult to use without visible reasoning traces, since detailed chain-of-thought is now gated behind a beta feature that shows only status-line updates.
Denser writing style. Anthropic’s own migration documentation lists denser sentences, fewer paragraph breaks, less use of bold headers, and unmarked quotations or summaries as known behavioral differences in Opus 5.1. Feedback is split: some find the output less “AI-sounding” and more direct, others find long-form documentation choppy and harder to follow.
Should you use Opus 5.1 or Opus 5 by default?
Anthropic itself now recommends Opus 5 as the default model, reserving Opus 5.1 for demanding reasoning tasks and long-horizon agentic work. Community sentiment partly agrees: several users report Opus 5 performing as well or better than Opus 5.1 for general tasks other than creative programming, while paying roughly double the cost for what they describe as minimal improvement.
Where Opus 5.1 clearly earns its price is in agentic and long-horizon coding work. The Claude Code team describes it as their best model for coding, data analysis, computer use, and extended agent runs, noting it progresses further into long tasks before needing user input and communicates more clearly when stuck. On a code review evaluation covering 45 tasks and 105 known issues, recall stayed roughly flat (61% versus 61.9% for Opus 5), but precision rose from 32.8% to 37.3%, total comments dropped 34%, and nitpicks dropped 70%. The tradeoff: reviews took 49% longer, and raising the reasoning effort setting to “high” actually made recall worse (57.1%), suggesting more thinking time doesn’t automatically improve review quality.
Frequently Asked Questions
What is the knowledge cutoff for Claude Opus 5.1?
Anthropic lists the knowledge cutoff as June 2026 in the model’s benchmark documentation.
Can you turn off extended thinking in Opus 5.1?
No. Thinking is always on and adaptive in Opus 5.1. Default effort is set to high in Claude Code and medium in Claude.ai and Co-Work, and there is no option to disable it entirely.
Why did my Opus 5.1 bill go up even though cache reads got cheaper?
Because cache write pricing didn’t change ($12.50 per million for 5-minute cache, $20 per million for 1-hour cache), and Opus 5.1 tends to perform more whole-file rewrites rather than targeted edits, generating more cache writes per task. On agentic workloads, cache writes can represent well over half the total bill even though they’re a small fraction of total tokens processed.
Does Opus 5.1 replace the need for Opus 5?
Not according to Anthropic’s own guidance, which recommends Opus 5 as the general default and Opus 5.1 for demanding reasoning or long-horizon agentic work specifically. Independent testers and community feedback broadly support keeping both in rotation depending on task type.
What is Claude Mythos 5.1?
Mythos 5.1 is the same underlying model as Opus 5.1 released the same day, but with safety measures removed. It is only available to vetted organizations within the United States and is not part of the general public release.