Qwen3.8-9B Distill by Empero: Benchmarks and Local Setup
Empero distilled Qwen3.8's reasoning into a 9B dense model. Here's what the MMLU and GSM8K benchmarks show, and how to run it locally.
What is Qwen3.8-9B Distill?
Qwen3.8-9B Distill is a full-parameter fine-tune released by Empero that transfers reasoning behavior from a much larger teacher model, Qwen3.8 2.4T A95B, into the smaller Qwen3.5-9B architecture. It was trained on roughly 70,000 curated chain-of-thought traces covering math, code, general reasoning, instruction following, and tool use. The result is a 9B dense model that runs on a single GPU but scores dramatically higher on MMLU than its own base model.
TL;DR
- Empero distilled a frontier-scale teacher, Qwen3.8 2.4T A95B, into a dense 9B student built on the Qwen3.5-9B architecture, using off-policy supervised fine-tuning rather than reinforcement learning.
- The training set was about 70,000 quality-filtered teacher traces, not synthetic self-generated reasoning, spanning math, code, instruction following, and tool use.
- MMLU accuracy jumped from 0.546 to 0.751 on flexible-extract scoring, a 20.5 point gain, with an even larger jump on strict-match (0.251 to 0.511).
- GSM8K scores dipped slightly, from 0.885 to 0.870 flexible and 0.875 to 0.850 strict, showing the distillation traded a sliver of grade-school math accuracy for broad knowledge and reasoning gains.
- The model is a full fine-tune, meaning every parameter was updated, not an adapter or LoRA layered on top of the base.
- It keeps the Qwen3.5 context window of 262,144 tokens and supports native function calling without a wrapper.
- Running it well requires specific kernels, including flash-linear-attention and a CUDA-matched causal_conv1d build, because the base architecture uses Gated DeltaNet linear-attention layers.
How was the distillation built?
Empero started with Qwen3.5-9B as the student architecture and Qwen3.8 2.4T A95B as the teacher. Instead of having the student generate its own rollouts and refining them through reinforcement learning, the team used off-policy distillation: they collected around 70,000 chain-of-thought traces generated directly by the teacher model, filtered them for quality, and used them as supervised fine-tuning data.
The trace mix was deliberately weighted toward mathematics and competitive programming, domains where the model card notes distillation tends to produce the biggest gains at this parameter scale. Every trace includes a <think> block, and the student learned to open its own answers the same way, mimicking the teacher’s reasoning style rather than developing an independent one.
This is a full-parameter fine-tune. Every weight in the 9B model was updated during training, using Hugging Face’s TRL and Transformers libraries. That distinguishes it from lighter-weight adapter methods like LoRA, which only touch a small subset of parameters and typically preserve more of the base model’s original behavior.
What do the benchmark numbers actually show?
Empero evaluated both the base Qwen3.5-9B and the distilled Qwen3.8-9B using EleutherAI’s lm-evaluation-harness, applying identical settings to both models so the comparison is apples to apples. Both models were treated as reasoning models and scored using chain-of-thought protocols: gsm8k_cot for grade-school math and mmlu_flan_cot_zeroshot for the 57-subject MMLU suite (roughly 1,700 questions).
The headline result is MMLU. Flexible-extract accuracy, which is the primary metric, rose from 0.546 in the base model to 0.751 in the distilled version, a gain of over 20 percentage points. Strict-match accuracy, which requires the answer to be formatted exactly right, rose even more sharply: from 0.251 to 0.511, more than doubling. That combination suggests the distilled model didn’t just get smarter, it also got much better at following the expected answer format, likely because the teacher traces reinforced consistent structure.
GSM8K tells a different story. Flexible-extract accuracy actually dropped slightly, from 0.885 to 0.870, and strict-match dropped from 0.875 to 0.850. Both declines are modest (1.5 and 2.5 points), but they indicate the distillation wasn’t a free lunch across every benchmark. It looks like the trace mix, weighted toward broader reasoning, code, and general knowledge, came at a small cost to narrow grade-school arithmetic performance, which the base model was already strong at.
Read together, the two benchmarks paint a specific picture: this distillation is a targeted trade. It buys substantial breadth and reasoning-format reliability on a wide-knowledge test like MMLU, at the cost of a small amount of specialized math accuracy on GSM8K.
Why does MMLU improve so much while GSM8K barely moves?
MMLU spans 57 subjects, from law and medicine to physics and history, and rewards a model that can reason carefully across broad, varied knowledge domains and communicate answers in a consistent format. Because the teacher, Qwen3.8 2.4T A95B, is a vastly larger model with much deeper general knowledge, its chain-of-thought traces likely modeled better-structured reasoning steps across many of these subjects, and the student picked up both the knowledge and the formatting discipline. The big strict-match jump (0.251 to 0.511) supports this: the student learned to answer in the exact form the grader expects, not just to reason correctly.
Remy doesn't build the plumbing. It inherits it.
Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something.
Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want.
GSM8K is narrower. It’s a set of grade-school math word problems, and the Qwen3.5-9B base was already performing well on it (0.885 flexible). There’s less room for a teacher’s broader reasoning traces to help, and if the trace mix shifted training emphasis toward code and general reasoning over pure arithmetic drills, a small regression on this specific benchmark is a plausible side effect. The model card is explicit that the trace mix was “deliberately weighted toward hard math and competitive programming,” which is not the same skill set as basic word-problem arithmetic.
Is Qwen3.8-9B Distill worth deploying?
For teams that want a single-GPU model with meaningfully stronger cross-domain reasoning than the Qwen3.5-9B base, the MMLU gains are hard to ignore. A 20-point jump in flexible-extract accuracy is a large move for a same-size model swap, and the license (Apache-2.0, inherited from the base) makes it usable in commercial contexts without additional negotiation.
The trade-offs to weigh: GSM8K performance regresses slightly, so if your use case leans heavily on simple arithmetic word problems, the base model may still be marginally better there. The model also inherits the teacher’s tendency toward long deliberation, according to Empero’s own best-practices notes, meaning it sometimes over-thinks easy questions. Every response opens with a <think> block that needs to be parsed out before showing output to end users, so downstream applications need to handle that stripping step.
There’s also a hardware wrinkle. Qwen3.5’s architecture uses Gated DeltaNet linear-attention layers, which need the flash-linear-attention and causal_conv1d kernel packages to run efficiently. Without them, the model falls back to slow, memory-heavy PyTorch operations. That’s a real setup cost worth budgeting for before assuming a smooth “pip install and go” deployment.
How do you run it locally?
The model is distributed in Hugging Face Transformers format and works with Transformers, vLLM, SGLang, and other runtimes that support the Qwen3.5 architecture. A minimal setup loads the model with AutoModelForCausalLM and AutoTokenizer, applies the chat template, and generates with the recommended sampling settings: temperature=0.6, top_p=0.95, top_k=20. Empero explicitly warns against greedy decoding on long generations, calling it a known repetition-loop failure mode for reasoning models in this class.
Because the model tends to produce long chain-of-thought output before its final answer, Empero recommends allowing generous generation length, up to 16,384 new tokens, and stripping the <think>...</think> span before showing results to users. You’ll also need a recent Transformers release with Qwen3.5 support and the linear-attention kernel packages installed to avoid a major performance hit.
Frequently Asked Questions
What base model does Qwen3.8-9B Distill use?
It’s built on Qwen/Qwen3.5-9B, developed by the Alibaba Qwen team, and inherits that model’s architecture, including its 262,144-token native context length and Gated DeltaNet linear-attention layers.
What teacher model was used for distillation?
Empero used Qwen3.8 2.4T A95B as the teacher, generating around 70,000 chain-of-thought traces that were quality-filtered and used to supervise the fine-tuning of the 9B student.
Does the distilled model support function calling?
Yes. It supports native function calling per Qwen3.5’s specification, so no wrapper or tool-specific fine-tune is required to use it for agentic or tool-use workflows.
Why did GSM8K scores go down slightly after distillation?
The exact cause isn’t detailed in Empero’s documentation, but the trace mix was weighted toward broader math, code, and general reasoning rather than narrow grade-school arithmetic, which likely explains the small 1.5 to 2.5 point drop on GSM8K even as MMLU improved substantially.
What hardware or software is needed to run it efficiently?
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
You need a recent version of Hugging Face Transformers with Qwen3.5 support, plus the flash-linear-attention and a CUDA-matched causal_conv1d build. Without these kernels, the linear-attention layers fall back to slow, memory-intensive PyTorch operations.
