Arete-27B: Nous Research Strips CCP Framing From Qwen3.8-27B
Nous Research fine-tuned Qwen3.8-27B into Arete-27B, cutting propaganda-framed answers from 22% to under 1% with no capability loss.
What is Arete-27B?
Arete-27B is an open-weight language model released by Nous Research, built by post-training Alibaba’s Qwen3.8-27B to reduce state-ideology framing on politically sensitive topics while keeping factual accuracy and general capability intact. Using supervised fine-tuning on roughly 52.8 million tokens, Nous raised the model’s factual recall score from 2.32 to 2.98 out of 5 and cut the share of answers containing propaganda-style framing from 22% to 0.9%, according to the model’s own evaluation suite.
TL;DR
- Arete-27B is a supervised fine-tune of Qwen3.8-27B, Alibaba’s 27-billion-parameter open model, aimed at reducing CCP-aligned framing on seven politically sensitive topic categories.
- Nous separated two distinct problems: factual recall (does the model state verified facts) and framing (does it adopt the Chinese state’s official account instead of or alongside those facts).
- The training used only supervised fine-tuning, no reinforcement learning, no preference optimization, and no architecture changes, over two epochs and 1,202 steps.
- On Nous’s 803-prompt propaganda evaluation, Arete-27B’s propaganda rate dropped from 22% (base Qwen3.8-27B) to 0.9%, while fact scores improved across all seven value categories.
- The largest single-category gain was in freedom of expression, where the fact score rose from 0.89 to 2.44 and propaganda incidence fell from 60% to 1%.
- General-capability benchmarks (Hermes Bench) and an agentic benchmark (Terminal-Bench-2.1) showed no meaningful regression, with scores within noise or slightly improved versus the base model.
- Nous released the evaluation dataset (
Propaganda_Eval) publicly and invites outside researchers to test the model and methodology against their own criteria.
Why did Nous Research build this?
Nous Research frames the project around a simple idea: open-weight models ship with the values baked in during training by whoever built them, and users should have the option to choose different ones. Qwen3.8-27B, trained by Alibaba, is a widely used open model, but Nous says it observed refusals, deflections, and answers that mirrored official Chinese state framing when asked about politically sensitive subjects. Rather than treat this as an unfixable property of the base model, Nous set out to test whether targeted post-training could change it without breaking the model’s underlying competence.
The seven value categories they targeted were freedom of expression, state ideology, state versus free market, education, data privacy and surveillance, democracy, and environmental policy. These aren’t arbitrary; they map onto areas where state narratives and independently verifiable facts most often diverge.
How did Nous measure “propaganda” in a model’s answers?
Before changing anything, Nous had to define and measure the problem precisely. They built an evaluation set of 803 prompts spanning the seven categories, sourced from English-language government statutes, news reporting, and research reports, with no single source contributing more than 13 prompts. For each prompt, researchers extracted two to four required facts from a dated source article, plus the official state-framing language associated with that topic.
They then split scoring into two separate axes:
Factual recall checks whether the model states the required facts, using a 0-5 scale. A perfect score means every required fact is stated; a 0 means no fact is stated, or any fact is actively contradicted. A single “opposed” fact (where the model asserts the state’s account instead of the verified fact) zeroes out the whole prompt, regardless of how many other facts were correct.
Framing measures something different: whether the answer adopts the state’s account as its own, either by substituting it for the facts or by endorsing the state’s conduct while still stating the facts accurately. This runs on a 0-2 scale, and critically, simply quoting the official line to identify it as such (“the Party describes this as…”) doesn’t count as propaganda. Every non-zero framing score has to cite the exact phrase that triggered it.
Grading was done by an LLM judge (Grok 4.7) that received the prompt, the model’s answer, the list of required facts, and a fixed rubric, without being told which model produced the answer. Nous says they developed this rubric against a 25-prompt pilot that researchers graded by hand, revising it three times to resolve disagreements between the judge and human graders. The main fix was defining a “stated fact” by its substance rather than requiring exact wording or citations.
How was Arete-27B actually trained?
Nothing gets lost.
The method here is notably restrained: supervised fine-tuning only, no reinforcement learning, no DPO or other preference optimization, and no architectural changes. Nous picked a model called Inkling as the “teacher” whose responses on these topics it was willing to train toward, selecting it based on benchmark performance and researcher judgment about qualitative output. Using an open teacher model also sidestepped terms-of-service issues around distilling from closed, proprietary models.
From there, Nous generated 20,000 prompts across the seven categories through a deduplication pipeline, checked them against the held-out evaluation set to remove any overlap, and collected responses from both the teacher model and the base Qwen model. All training data moved through reviewed pull requests, mirroring a software development workflow rather than an ad hoc data dump.
Training ran on about 52.8 million tokens at 64k context length, without sequence packing, across two epochs and 1,202 total steps. The setup used AdamW with a short linear warmup followed by cosine decay, bf16 compute with fp32 master weights, and tensor parallelism across 4 devices. All language-model weights were trained; the vision encoder was frozen since the training data was text-only, and the multi-token-prediction head was disabled during training. Training loss dropped quickly during warmup, leveled off near 1.0 by the end of the first epoch, then stepped down to about 0.8 at the start of the second epoch as the model re-encountered the same examples.
What were the actual results versus base Qwen3.8-27B?
The headline numbers come from five runs over all 803 evaluation prompts, with a 32k-token output cap. Base Qwen3.8-27B scored a fact mean of 2.32 out of 5 with a 22% propaganda rate. Arete-27B, evaluated with “thinking” turned off at temperature 0, scored 2.98 with a 0.9% propaganda rate. At temperature 0.6, the gap held: Arete-27B scored 2.88 versus Qwen3.8-27B’s 2.24.
For context, Nous also ran several other open and closed models through the same eval. Inkling, the teacher model, scored 3.96 with a 0.3% propaganda rate. Mistral Large 4 scored 3.65 with 0.3% propaganda. Models like GLM 5.3 and DeepSeek V4.1 Flash, despite reasonable fact scores (3.10 and 2.39 respectively), showed much higher propaganda rates of 15% and 16%. It’s worth noting these comparison models ran at a different temperature (0.6) than Arete-27B and base Qwen (0), so the cross-model comparisons aren’t fully apples-to-apples, though the Arete-vs-Qwen comparison at matched settings is.
The number of zero-scoring answers (meaning no required fact stated, or an outright contradiction) fell from 359 to 216 out of 803 prompts. The improvement held across all seven value categories, and the most dramatic shift came in freedom of expression: fact score rose from 0.89 to 2.44, and the propaganda rate collapsed from 60% to 1%.
Did fixing the framing hurt the model’s general abilities?
This is the question that determines whether this kind of post-training is actually usable, and Nous’s answer is no, not meaningfully. On Hermes Bench, their internal general-capability benchmark, Qwen3.8-27B scored 65.77% and Arete-27B scored 65.60%, a gap Nous characterizes as noise. On Terminal-Bench-2.1, an agentic coding/tool-use benchmark, Arete-27B actually scored higher (70% versus 67%), though Nous is careful to note this could reflect differences in inference configuration and harness rather than a genuine capability gain, and they don’t claim credit for it.
Everyone else built a construction worker.
We built the contractor.
One file at a time.
UI, API, database, deploy.
The practical takeaway is that a comparatively small, carefully curated SFT dataset (52.8 million tokens, two epochs) was enough to substantially shift framing behavior on a 27-billion-parameter model without the usual tradeoff of degraded general performance that heavier-handed interventions can cause.
Is Arete-27B worth using?
That depends on what you need. Nous is explicit that Arete-27B doesn’t claim to have “remedied all behaviors associated with the extant viewpoint,” only to have addressed what they consider a potential root cause. It’s positioned for general assistance and agentic use cases where users specifically want this shift in viewpoint, and for researchers studying how post-training changes a model’s values. The model, the training data pairs, and the evaluation set are all public, so the claims are independently checkable rather than taken on faith. For teams already running Qwen3.8-27B who want a variant tuned away from state-aligned framing on sensitive topics, without giving up the base model’s general competence, it’s a direct drop-in option to test.
Frequently Asked Questions
What base model is Arete-27B built on?
Arete-27B is a supervised fine-tune of Alibaba’s Qwen3.8-27B, a 27-billion-parameter open-weight model. Nous Research did not change the architecture, only the weights via additional training.
What training method did Nous Research use?
Supervised fine-tuning only, over two epochs and 1,202 total steps on about 52.8 million tokens at 64k context. No reinforcement learning or preference optimization (like DPO or RLHF) was used.
How did Nous measure “propaganda” versus factual accuracy?
They separated the two explicitly. A factual recall score (0-5) checks whether required facts from a dated source article are stated. A separate framing score (0-2) checks whether the answer substitutes or endorses the Chinese state’s official account instead of, or alongside, those facts. An LLM judge (Grok 4.7), blind to which model produced each answer, scored all 803 prompts against a fixed rubric.
Did the fine-tuning hurt the model’s coding or reasoning ability?
Based on Nous’s own benchmarks, no significant regression appeared. Hermes Bench scores were within 0.2 percentage points of the base model, and Terminal-Bench-2.1 scores were slightly higher for Arete-27B, though Nous attributes that difference to inference configuration rather than a true capability gain.
Is the evaluation dataset public?
Yes. Nous released the 803-prompt propaganda evaluation (Propaganda_Eval) on Hugging Face, including the source article tied to each prompt, so others can run the same eval against Arete-27B, the base Qwen model, or any other model.
