Nanbeige4.3-33B Benchmarks: How It Stacks Up on Agentic Coding
Nanbeige4.3-33B's Terminal-Bench, SWE-Bench Pro, and cowork benchmark scores against Qwen3.8-27B, Muse-Glimmer-30B, and Gemma4-31B.
What is Nanbeige4.3-33B and how does it perform?
Nanbeige4.3-33B is an open-weight language model from the Nanbeige team, released under Apache 2.0, sized at roughly 33 billion parameters. Across a set of agentic coding and professional cowork benchmarks, it posts the top score among four open models near the 30B scale, beating Qwen3.8-27B, Muse-Glimmer-30B, and Gemma4-31B on most tests. It leads by wide margins on tasks like Terminal-Bench 4.0 and DeepSWE, trails Qwen3.8-27B slightly on SWE-Bench Pro and GDPval, and holds a near-tie on general reasoning benchmarks like GPQA Diamond and AIME 2026.
TL;DR
- Nanbeige4.3-33B is positioned as a teacher model, built first to anchor a distillation pipeline that will later produce smaller agentic models, including a planned Nanbeige4.3-3B.
- On Terminal-Bench 2.1, it scores 80.9 versus 73.0 for Qwen3.8-27B, 61.8 for Muse-Glimmer-30B, and 47.2 for Gemma4-31B, the widest clean sweep in the coding category.
- On Terminal-Bench 4.0, the gap widens further: 23.8 for Nanbeige4.3-33B against single digits or near-zero for every competitor, suggesting the harder benchmark separates models more sharply.
- It does not win everywhere: SWE-Bench Pro goes to Qwen3.8-27B (61.7 vs 60.3), and GDPval also favors Qwen3.8-27B (86.4 vs 84.8), showing the lead isn’t universal.
- In the agentic cowork category, covering tasks like JobBench, Automation Bench, and Toolathlon-Verified, Nanbeige4.3-33B tops nearly every listed benchmark, including a notably large edge on YC-bench’s dollar-value scoring.
- Reasoning benchmarks (GPQA Diamond, AIME 2026, AA-LCR) are close across all four models, with Gemma4-31B actually taking GPQA Diamond, indicating raw reasoning gains are smaller than agentic task gains.
- The model supports a 512K token context window and ships with recommended inference settings split by use case (agentic tasks vs. reasoning/chat).
How does Nanbeige4.3-33B compare to Qwen3.8-27B, Muse-Glimmer-30B, and Gemma4-31B?
The model card presents a head-to-head table across 20 benchmarks split into three categories: agentic coding, agentic cowork, and reasoning. All four models were evaluated at their highest thinking effort, using identical inference hyperparameters and harnesses for each benchmark, which makes the comparison more apples-to-apples than most cross-lab benchmark claims.
In agentic coding, Nanbeige4.3-33B wins 7 of 8 listed benchmarks. The only loss is SWE-Bench Pro, where Qwen3.8-27B edges ahead by 1.4 points (61.7 vs 60.3). Everywhere else, the margins are substantial: on DeepSWE, Nanbeige4.3-33B scores 65.5 against Qwen3.8-27B’s 42.5, and against single-digit scores for Muse-Glimmer-30B and Gemma4-31B. That kind of gap suggests DeepSWE rewards agentic planning and tool use in a way the smaller or differently-trained competitors struggle with.
In agentic cowork, which covers tasks closer to office and knowledge work (scheduling, enterprise operations, tool-calling dialogues), Nanbeige4.3-33B wins 7 of 9 benchmarks. GDPval goes to Qwen3.8-27B, and EnterpriseOps-Gym is close (41.0 vs 40.2). The standout is YC-bench, scored in dollar terms rather than percentages, where Nanbeige4.3-33B posts $1,025,437 against $641,290 for Qwen3.8-27B, nearly 60% higher.
Reasoning benchmarks tell a different story. GPQA Diamond, AIME 2026, and AA-LCR all show the four models within a few points of each other, and Gemma4-31B actually posts the highest GPQA Diamond score (91.3) despite finishing last on almost every agentic benchmark. This pattern, strong agentic performance paired with merely competitive raw reasoning, lines up with Nanbeige’s own framing: the model is built and tuned specifically for agent workflows, not just benchmark-maximized general reasoning.
What is Terminal-Bench and why does the score gap matter?
Terminal-Bench measures how well a model can complete tasks inside a terminal environment: navigating file systems, running commands, debugging, and completing multi-step technical workflows autonomously. It’s become a common proxy for whether a model can act as a coding agent rather than just a code-completion assistant.
The model card reports two versions. Terminal-Bench 2.1, evaluated using the Terminus 2 harness, shows Nanbeige4.3-33B at 80.9, ahead of Qwen3.8-27B (73.0), Muse-Glimmer-30B (61.8), and Gemma4-31B (47.2). Terminal-Bench 4.0, a harder revision evaluated with mini-swe-agent v2.4.6, compresses everyone’s scores dramatically: Nanbeige4.3-33B drops to 23.8, but the competitors drop much further, to 6.3, 1.6, and 0.0 respectively.
That near-zero score for Gemma4-31B on Terminal-Bench 4.0 is worth noting. It implies the newer benchmark version introduces task complexity or evaluation strictness that most ~30B models simply can’t handle yet, and that the gap between “good” and “state of the art” at this parameter scale is currently large on harder agentic tests.
What is SWE-Bench Pro and where does Nanbeige4.3-33B fall short?
SWE-Bench Pro is a professional-grade variant of the SWE-Bench family, testing models on realistic software engineering tasks pulled from real repositories rather than synthetic problems. It’s one of the more demanding tests in the table, and it’s also the one benchmark in the coding category where Nanbeige4.3-33B doesn’t lead.
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
Qwen3.8-27B scores 61.7 versus Nanbeige4.3-33B’s 60.3, a gap of 1.4 points, small enough to be within normal benchmark variance but notable because it’s a repeatable pattern: Qwen3.8-27B also wins GDPval, a benchmark scored with revised rubrics meant to reflect professional task quality. Both benchmarks emphasize precision on complex, realistic work rather than breadth across many short agentic tasks, which may explain why Qwen3.8-27B holds an edge there even while trailing broadly elsewhere.
Is Nanbeige4.3-33B worth using for agentic coding work?
For teams building coding agents or automation pipelines at the ~30B parameter scale, Nanbeige4.3-33B’s benchmark profile makes a reasonably strong case. It leads on Terminal-Bench, DeepSWE, SWE-Atlas-QnA, NL2Repo, Scicode, and SkillsBench, covering a broad span of coding-adjacent tasks rather than just one narrow benchmark. The cowork category results (JobBench, Automation Bench, Toolathlon-Verified, DeepPlanning) suggest it also generalizes to non-coding agentic work like task automation and multi-step planning.
The caveats are real, though. It’s apache-2.0 licensed and open-weight, but downloads were at zero at the time of its model card listing, meaning there’s no independent usage track record yet. The benchmark table comes entirely from Nanbeige’s own evaluation, run with harnesses the team selected (Terminus 2, mini-swe-agent, Claude Code as a harness and judge in some cases), and no third-party reproduction is cited. And on the benchmark that arguably matters most for production software engineering, SWE-Bench Pro, it’s not actually the top scorer.
The model does ship with practical deployment detail: a 512K token context window, Transformers/SGLang/vLLM support, and distinct recommended settings for agentic tasks (temperature 1.0, top_p 0.95, 65,536 max tokens) versus reasoning and chat tasks (temperature 0.6, top_p 0.95, 262,144 max tokens). That level of specificity is useful for anyone trying to actually run it rather than just read about it.
Frequently Asked Questions
What is Nanbeige4.3-33B?
It’s an open-weight, roughly 33-billion-parameter language model from the Nanbeige team, released under an Apache 2.0 license and built as the flagship “teacher” model for a planned family of smaller distilled agentic models.
How big is Nanbeige4.3-33B and what context length does it support?
It’s built on a Llama-architecture base model (Nanbeige4.3-33B-Base) and supports a context window of up to 524,288 tokens (512K), according to its model card.
What benchmarks does Nanbeige4.3-33B lead on?
It tops most agentic coding and cowork benchmarks reported, including Terminal-Bench 2.1, Terminal-Bench 4.0, DeepSWE, SWE-Atlas-QnA, NL2Repo, Scicode, SkillsBench, Automation Bench, JobBench, Claw-Eval, Toolathlon-Verified, DeepPlanning, and YC-bench.
Where does Nanbeige4.3-33B lose to competitors?
Qwen3.8-27B scores higher on SWE-Bench Pro and GDPval. Gemma4-31B scores slightly higher on GPQA Diamond, a pure knowledge/reasoning benchmark rather than an agentic one.
How do you run Nanbeige4.3-33B?
The model card documents deployment through Transformers, SGLang, and vLLM, with sample commands for each, plus recommended temperature and token settings that differ for agentic/tool-use tasks versus general reasoning and chat tasks.
