Nanbeige4.3-33B Benchmarks: How It Beats Qwen3.8-27B and Gemma4-31B
Nanbeige4.3-33B tops Terminal-Bench, DeepSWE and JobBench among ~30B open models, beating Qwen3.8-27B and Gemma4-31B on most agentic tasks.
What is Nanbeige4.3-33B and why does it matter?
Nanbeige4.3-33B is an open-weight language model from the Nanbeige team, built around 33 billion parameters, positioned as the flagship “teacher” model in the Nanbeige4.3 series. Its stated purpose is narrower than a general chatbot: excel at agentic coding and professional cowork tasks, then use that capability to distill smaller models later, including a planned Nanbeige4.3-3B. Against three other roughly 30B-class open models (Qwen3.8-27B, Muse-Glimmer-30B, and Gemma4-31B), Nanbeige4.3-33B leads on most benchmarks the team published, particularly in coding agent evaluations and workplace-style task suites.
TL;DR
- Nanbeige4.3-33B leads most agentic benchmarks against Qwen3.8-27B, Muse-Glimmer-30B, and Gemma4-31B, winning 14 of the 19 benchmarks in the released comparison table.
- The gap widens sharply on Terminal-Bench 4.0, where Nanbeige4.3-33B scores 23.8 versus 6.3 for Qwen3.8-27B and near-zero for the other two models.
- DeepSWE shows the same pattern, with Nanbeige4.3-33B at 65.5 against Qwen3.8-27B’s 42.5 and single-digit scores from Muse-Glimmer-30B and Gemma4-31B.
- Qwen3.8-27B still wins on SWE-Bench Pro and GDPval, meaning Nanbeige4.3-33B isn’t uniformly dominant, especially on tasks closer to professional deliverable quality.
- Reasoning benchmarks are close across the board, with Gemma4-31B actually topping GPQA Diamond despite trailing badly on agentic tasks.
- The model supports a 512K token context window and ships with configurable chat templates for thinking mode, tool use, and multi-turn agent workflows.
- It runs on standard open-source serving stacks including Transformers, SGLang, and vLLM, with tensor parallelism recommended for deployment.
How does Nanbeige4.3-33B perform on agentic coding benchmarks?
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
The model card groups benchmarks into three categories: agentic coding, agentic cowork, and reasoning. Agentic coding is where Nanbeige4.3-33B shows its clearest advantage.
On Terminal-Bench 2.1, evaluated using Terminus 2, it scores 80.9 against Qwen3.8-27B’s 73.0, Muse-Glimmer-30B’s 61.8, and Gemma4-31B’s 47.2. That’s already a meaningful lead, but the newer Terminal-Bench 4.0 (run with mini-swe-agent v2.4.6) shows a much starker contrast: Nanbeige4.3-33B hits 23.8, more than triple Qwen3.8-27B’s 6.3, while Muse-Glimmer-30B manages 1.6 and Gemma4-31B scores 0.0. Terminal-Bench 4.0 appears to be a harder, more recent version of the benchmark where most ~30B models struggle, and Nanbeige4.3-33B’s relative lead grows rather than shrinks.
DeepSWE follows the same shape: 65.5 for Nanbeige4.3-33B versus 42.5 for Qwen3.8-27B, and single digits for the other two models. On SWE-Atlas-QnA, judged by GPT-6 Sol against 1,304 task-specific rubrics, Nanbeige4.3-33B scores 50.8 against Qwen3.8-27B’s 41.9.
The one coding benchmark where it loses is SWE-Bench Pro, where Qwen3.8-27B edges ahead at 61.7 versus 60.3. That’s a narrow gap and the only coding benchmark in the table where Nanbeige4.3-33B isn’t first, but it’s worth flagging because SWE-Bench Pro is one of the more widely cited software engineering benchmarks in the field.
On NL2Repo (evaluated using Claude Code as the harness), Scicode, and SkillsBench, Nanbeige4.3-33B again leads, though by smaller margins than the Terminal-Bench and DeepSWE results.
How does it compare on agentic cowork tasks?
“Agentic cowork” benchmarks aim at tasks closer to real office or business work: planning, tool orchestration, automation, and professional judgment calls. This category includes nine benchmarks in the released comparison, and Nanbeige4.3-33B wins seven of them.
Automation Bench (600 public tasks, v1.0.6) shows Nanbeige4.3-33B at 54.2 against Qwen3.8-27B’s 38.3, a wider gap than most of the coding benchmarks. JobBench, evaluated with a Claude Code harness equipped with a search tool, shows 44.8 versus 34.5. The model card specifically highlights that on JobBench, which spans 35 distinct occupations, Nanbeige4.3-33B outperforms the baseline in most of them, suggesting the advantage isn’t concentrated in one or two task types but spreads across varied professional domains.
DeepPlanning shows one of the largest gaps in the entire table: 63.3 for Nanbeige4.3-33B against 49.2 for Qwen3.8-27B. YC-bench, scored in dollar terms (likely simulating business outcomes or valuations), shows Nanbeige4.3-33B at $1,025,437 against Qwen3.8-27B’s $641,290, Muse-Glimmer-30B’s $513,978, and Gemma4-31B’s $265,152.
GDPval is the exception. Qwen3.8-27B wins here, 86.4 to 84.8, a close contest but a loss nonetheless for Nanbeige4.3-33B. EnterpriseOps-Gym and Tau3-Bench are also close, with Nanbeige4.3-33B holding narrow leads (41.0 vs 40.2, and 48.8 vs 48.5 respectively) that look more like statistical ties than clear wins.
Does Nanbeige4.3-33B win on general reasoning too?
Not decisively, and this is the honest caveat in the data. The reasoning category includes GPQA Diamond, AIME 2026, and AA-LCR. Gemma4-31B actually tops GPQA Diamond at 91.3 versus Nanbeige4.3-33B’s 89.5, despite Gemma4-31B trailing badly on almost every agentic benchmark. That split is worth noting: strong performance on graduate-level science questions doesn’t predict strong performance on terminal-based coding agents or multi-step planning tasks.
Seven tools to build an app. Or just Remy.
Editor, preview, AI agents, deploy — all in one tab. Nothing to install.
Nanbeige4.3-33B does lead AIME 2026 (95.2 vs Qwen3.8-27B’s 94.5 and Muse-Glimmer-30B’s 94.7, a near three-way tie) and AA-LCR (84.0 vs 82.0). But the margins here are much tighter than in the agentic categories, reinforcing that this model’s differentiation is specifically in agent-style, tool-using, multi-step workflows rather than raw knowledge or math reasoning.
Is Nanbeige4.3-33B worth deploying for agentic workloads?
For teams building coding agents or automation pipelines and evaluating options in the ~30B parameter range, the published benchmarks suggest Nanbeige4.3-33B is a strong candidate, particularly for terminal-based coding tasks and multi-step business workflows. The Terminal-Bench 4.0 and DeepSWE results are the most striking because the performance gap over Qwen3.8-27B, the closest competitor, is large rather than marginal.
That said, a few practical points matter before committing to it. First, Qwen3.8-27B remains competitive or better on SWE-Bench Pro and GDPval, both benchmarks that lean toward professional deliverable quality rather than raw task completion, so teams whose use case maps closer to those evaluations should weigh that tradeoff. Second, all benchmark numbers come from the Nanbeige team’s own model card and testing harness; the team notes it used consistent inference hyperparameters and harnesses across all four models, which is good methodological practice, but independent third-party verification isn’t yet reflected in these figures.
Third, deployment is straightforward if you’re already running Transformers, SGLang, or vLLM. The model supports a 512K token context window and ships with a configurable chat template handling thinking mode, multi-turn tool preservation, and both XML and JSON tool-call formats (XML is the team’s recommendation for best tool-calling performance). Recommended inference settings differ by use case: temperature 1.0 and top_p 0.95 for agentic and tool-use tasks, versus temperature 0.6 for reasoning and chat tasks, with max new tokens scaling up to 262,144 for longer reasoning workloads.
Frequently Asked Questions
What size is Nanbeige4.3-33B?
It’s a 33 billion parameter open-weight model, positioned in the same class as Qwen3.8-27B, Muse-Glimmer-30B, and Gemma4-31B, all roughly 30B-parameter models compared in the same benchmark table.
What benchmark does Nanbeige4.3-33B win by the widest margin?
Terminal-Bench 4.0 shows the largest relative gap: 23.8 for Nanbeige4.3-33B against 6.3 for Qwen3.8-27B, 1.6 for Muse-Glimmer-30B, and 0.0 for Gemma4-31B. DeepSWE shows a similarly wide gap in absolute score difference.
Does Nanbeige4.3-33B beat Qwen3.8-27B on every benchmark?
No. Qwen3.8-27B leads on SWE-Bench Pro (61.7 vs 60.3) and GDPval (86.4 vs 84.8), both tasks oriented toward professional deliverable quality rather than raw agentic task completion.
What context length does Nanbeige4.3-33B support?
Up to 524,288 tokens (512K), according to the model’s official documentation.
What inference frameworks support Nanbeige4.3-33B?
The model card provides setup instructions for Transformers, SGLang, and vLLM, with tool-call parsing configured for Qwen3-style reasoning and tool-call formats.
