Nanbeige4.3-33B Benchmarks: Does This 33B Model Beat Qwen3.8-27B?
Nanbeige4.3-33B's benchmark scores on agentic coding and cowork tasks, compared against Qwen3.8-27B, Muse-Glimmer-30B, and Gemma4-31B.
What is Nanbeige4.3-33B?
Nanbeige4.3-33B is an open-weight language model released by the Nanbeige Team under an Apache 2.0 license, built as a “teacher agent” for distilling agentic capability into smaller models. On most of the agentic coding and cowork benchmarks published in its model card, it scores higher than three other roughly 30B-parameter open models: Qwen3.8-27B, Muse-Glimmer-30B, and Gemma4-31B. It supports a 512K token context window and is built for tool use, multi-step planning, and coding tasks rather than general chat.
TL;DR
- Nanbeige4.3-33B is positioned as a teacher model, meaning its primary purpose is anchoring a distillation pipeline for smaller models like a planned Nanbeige4.3-3B, not just standing alone as a chat assistant.
- It leads on most agentic coding benchmarks, including Terminal-Bench 2.1 (80.9 vs. 73.0 for Qwen3.8-27B), Terminal-Bench 4.0 (23.8 vs. 6.3), and DeepSWE (65.5 vs. 42.5).
- Qwen3.8-27B still wins on a few benchmarks, notably SWE-Bench Pro (61.7 vs. 60.3) and GDPval (86.4 vs. 84.8), showing the comparison isn’t a clean sweep.
- Agentic cowork scores favor Nanbeige4.3-33B broadly, with gains on Automation Bench, JobBench, DeepPlanning, and a notably large margin on YC-bench’s dollar-value scoring.
- Pure reasoning benchmarks are close across the board, with Gemma4-31B actually topping GPQA Diamond (91.3) despite trailing on agentic tasks.
- The model supports a 512K token context window and ships with recommended inference settings that differ for agentic tool use versus general reasoning and chat.
- It’s deployable today via Transformers, SGLang, or vLLM, with tensor parallelism and tool-call parsing built into the standard serving commands.
Remy doesn't write the code. It manages the agents who do.
Remy runs the project. The specialists do the work. You work with the PM, not the implementers.
How does Nanbeige4.3-33B compare to Qwen3.8-27B, Muse-Glimmer-30B, and Gemma4-31B?
The Nanbeige team ran all four models at their highest thinking effort, using identical inference hyperparameters and evaluation harnesses per benchmark, across three categories: agentic coding, agentic cowork, and general reasoning.
In agentic coding, Nanbeige4.3-33B leads on seven of eight benchmarks. The gaps are largest on the newer or harder tests: Terminal-Bench 4.0 (23.8 vs. 6.3 for Qwen, 1.6 for Muse-Glimmer, and 0.0 for Gemma4) and DeepSWE (65.5 vs. 42.5, 4.4, and 2.7 respectively). The one exception is SWE-Bench Pro, where Qwen3.8-27B edges ahead at 61.7 versus 60.3.
In agentic cowork, which covers things like automation, job-specific task completion, and multi-turn planning, Nanbeige4.3-33B again leads on most benchmarks: Automation Bench (54.2 vs. 38.3), JobBench (44.8 vs. 34.5), DeepPlanning (63.3 vs. 49.2), and the YC-bench dollar-value metric ($1,025,437 vs. $641,290 for Qwen, $513,978 for Muse-Glimmer, $265,152 for Gemma4). GDPval is the standout exception here, where Qwen3.8-27B scores 86.4 against Nanbeige’s 84.8.
On pure reasoning benchmarks (GPQA Diamond, AIME 2026, AA-LCR), the four models cluster tightly. Gemma4-31B actually posts the top GPQA Diamond score (91.3), even though it trails badly on agentic tasks. This split matters: a model’s raw reasoning score doesn’t predict how well it chains tool calls, writes code across a repo, or completes a multi-step professional task.
What do the agentic coding benchmarks actually measure?
The coding benchmarks in the card test different slices of real software engineering work, not just code generation:
- Terminal-Bench 2.1 and 4.0 evaluate a model’s ability to operate inside a terminal environment, using an agent harness (Terminus 2 for 2.1, mini-swe-agent v2.4.6 for 4.0) to complete multi-step command-line tasks.
- DeepSWE and SWE-Bench Pro test realistic software engineering fixes, similar in spirit to the original SWE-bench, where a model has to understand a codebase and produce a correct patch.
- SWE-Atlas-QnA grades answers against roughly 1,300 task-specific rubrics using an LLM judge (GPT-6 Sol at medium reasoning effort).
- NL2Repo evaluates turning natural-language specs into working repository code, run through the Claude Code harness.
- Scicode and SkillsBench round out the suite with scientific coding and broader skill-based tasks.
Nanbeige4.3-33B’s strongest relative margins show up on the newer, harder benchmarks (Terminal-Bench 4.0, DeepSWE), where all four models score lower overall, suggesting these tasks separate genuine agentic coding ability from benchmark-saturated performance.
What does “agentic cowork” mean in this context?
Agentic cowork benchmarks try to simulate working alongside or in place of a human professional on multi-step, often ambiguous tasks, rather than isolated coding problems. The suite includes:
- Automation Bench (600 publicly released tasks) and JobBench, which test task completion across a spread of professions. The model card notes Nanbeige4.3-33B outperforms the baseline on most of the 35 occupations evaluated in JobBench.
- GDPval, scored with revised rubrics, which aims to approximate real economic task value.
- Claw-Eval, run on its “General” subset of 157 single-turn user interaction tasks.
- Toolathlon-Verified, run with no token limit per model request, testing extended tool-use chains.
- DeepPlanning, EnterpriseOps-Gym, and Tau3-Bench, each run through their official evaluation repositories, covering planning depth and enterprise-style operational tasks.
- YC-bench, which scores outputs in dollar terms, keeping only the most recent 10 turns of dialogue as context before each call.
One coffee. One working app.
You bring the idea. Remy manages the project.
This is a notably broad evaluation surface for a 33B open model. The occupation-level breakdown on JobBench in particular signals an attempt to show generalization across job types rather than optimizing for a single flagship benchmark.
Is Nanbeige4.3-33B worth deploying over Qwen3.8-27B?
The honest answer depends on the task. If the use case is agentic coding (terminal operations, repo-level fixes, natural-language-to-code) or broad cowork automation, Nanbeige4.3-33B’s benchmark lead is consistent and sometimes large, especially on Terminal-Bench 4.0 and DeepSWE. If the priority is SWE-Bench Pro style patch generation or GDPval-style economic task value, Qwen3.8-27B currently scores slightly higher on those two specific benchmarks.
For pure reasoning tasks like scientific QA or math competitions, the four models are close enough (within a few points on GPQA Diamond, AIME 2026, and AA-LCR) that benchmark choice matters less than deployment constraints like context length, licensing, or inference cost.
Nanbeige4.3-33B’s Apache 2.0 license and support for standard serving stacks (Transformers, SGLang, vLLM) make it straightforward to self-host for teams already running open-weight models in production.
How do you run Nanbeige4.3-33B?
The model is available on Hugging Face and runs through Transformers, SGLang, or vLLM. It supports a configurable chat template with two key flags: enable_thinking, which turns reasoning traces on or off (on by default), and preserve_thinking, which controls whether reasoning from earlier turns in a conversation is retained. The Nanbeige team recommends setting preserve_thinking to False for general chat and True for multi-turn tool use, office tasks, and code-agent workflows, which matches how the model was actually benchmarked (with preserve_thinking=true).
For tool calling, the model supports both XML and JSON formats, with XML recommended for best performance. Context length tops out at 512K tokens. Recommended inference settings differ by use case: agentic and tool-use tasks call for temperature 1.0, top_p 0.95, and up to 65,536 new tokens, while reasoning and chat tasks use a lower temperature of 0.6 with up to 262,144 new tokens.
Serving at scale uses standard tensor-parallel setups. The SGLang and vLLM launch commands in the model card both use a tensor-parallel size of 4, a Qwen3-compatible reasoning parser, and a Qwen3-coder tool-call parser, suggesting the tokenizer and tool-calling format are deliberately compatible with existing Qwen-based tooling.
Frequently Asked Questions
What parameter size is Nanbeige4.3-33B?
It’s a 33 billion parameter model, built on a base model called Nanbeige4.3-33B-Base, released as safetensors weights under an Apache 2.0 license.
What is a “teacher agent” model?
In this context, a teacher agent is a larger, high-capacity model used to generate training signal (distillation data) for smaller models. Nanbeige4.3-33B is explicitly designed to anchor a distillation pipeline, with a smaller Nanbeige4.3-3B model planned to inherit its agentic capabilities.
Does Nanbeige4.3-33B beat Qwen3.8-27B on every benchmark?
No. It leads on most agentic coding and cowork benchmarks, but Qwen3.8-27B scores higher on SWE-Bench Pro (61.7 vs. 60.3) and GDPval (86.4 vs. 84.8), and the two models are essentially tied on Tau3-Bench.
What context length does Nanbeige4.3-33B support?
Up to 524,288 tokens (512K), according to the model card.
How was Nanbeige4.3-33B evaluated?
Plans first. Then code.
Remy writes the spec, manages the build, and ships the app.
All four compared models (Nanbeige4.3-33B, Qwen3.8-27B, Muse-Glimmer-30B, Gemma4-31B) were run at their highest thinking effort, using the same inference hyperparameters and evaluation harness per benchmark, across agentic coding, agentic cowork, and reasoning categories.
