Arete-27B: Nous Research's Fine-Tune Strips CCP Framing From Qwen3.8-27B
Nous Research fine-tuned Qwen3.8-27B to cut propaganda-framed answers from 22% to under 1%, with details on method and benchmark results.
What is Arete-27B?
Arete-27B is an open-weight language model from Nous Research, built by post-training Alibaba’s Qwen3.8-27B to reduce state-aligned framing on politically sensitive topics. Using supervised fine-tuning only, Nous raised the model’s factual recall score on its own propaganda evaluation from 2.32 to 2.98 out of 5, while cutting the share of answers containing propaganda-style framing from 22% to 0.9%. The model is released under Apache 2.0 on Hugging Face.
TL;DR
- Arete-27B is a 27-billion-parameter fine-tune of Qwen3.8-27B, trained by Nous Research to address refusals, deflections, and state-aligned framing the team observed on politically sensitive prompts.
- The training used supervised fine-tuning only, no reinforcement learning, no preference optimization, and no architecture changes, over roughly 52.8 million tokens at 64k context across two epochs and 1,202 steps.
- Nous built a held-out evaluation set of 803 prompts across seven value categories (freedom of expression, state ideology, state vs. free market, education, data privacy, democracy, and environmental policy), each checked against a dated source article for required facts.
- On that eval, propaganda-scoring answers dropped from 22% in the base Qwen3.8-27B model to 0.9% in Arete-27B, with gains in all seven categories, the largest being freedom of expression where the propaganda rate fell from 60% to 1%.
- The teacher model used for distillation was an open model called Inkling, chosen specifically to avoid “illicit distillation” restrictions tied to closed-model terms of use.
- General-capability and agentic benchmarks (Hermes Bench and Terminal-Bench-2.1) showed no meaningful regression, with Hermes Bench scores within 0.2 percentage points of the base model.
- Nous published both the model weights and the evaluation dataset (
Propaganda_Eval), inviting outside researchers to test the methodology themselves.
How does Arete-27B’s training process work?
Nous started from Qwen3.8-27B, a model trained by Alibaba, and ran supervised fine-tuning on roughly 20,000 prompts spanning seven value categories. The pipeline generated prompts, deduplicated them against the held-out evaluation set to prevent contamination, and collected responses from both a teacher model (Inkling) and the base Qwen model for comparison.
Training ran for two epochs over 1,202 total steps (601 per epoch), using the AdamW optimizer with a learning rate that warmed up linearly from 1e-6 to 1e-5 over the first 120 steps, then decayed on a cosine schedule back down to 1e-6. The batch size was 48, with bf16 compute and fp32 master weights. All language-model weights were trained; the vision encoder was frozen since the training data was text-only, and the multi-token-prediction head was disabled during training. No sequence packing was used.
Training loss dropped quickly during warmup, flattened near 1.0 by the end of epoch 1, then stepped down to about 0.8 at the start of epoch 2 (expected, since the model was seeing repeated examples), and stayed flat from there. Gradient norm started near 32 and settled around 1.4 once warmup finished.
Every batch of training data went through reviewed pull requests, the same workflow Nous uses for code changes, a detail that signals the team treated dataset construction with the same scrutiny as software.
Why did Nous Research build this model?
Nous Research frames the project around a simple premise: open-weight models let anyone reshape a model’s behavior, but the values a model ships with by default shouldn’t be the only option available to users. The team says it observed refusals, deflections, and CCP-framed answers from Qwen3.8-27B on politically sensitive topics, and set out to test whether targeted post-training could change that without wrecking the model’s general abilities.
Importantly, Nous draws a distinction between two different problems: factual recall (does the model state facts that can be corroborated by verified sources) and framing (does the model adopt the Chinese state’s account as its own). This split matters because a model could already state facts accurately while still wrapping them in state-approved framing, or vice versa. Nous built its evaluation specifically to separate these two failure modes rather than treating “bias” as a single blurry category.
The seven value categories they targeted: freedom of expression, state ideology, state versus free market, education, data privacy and surveillance, democracy, and environmental policy, were chosen as areas where state-aligned framing was most likely to show up in model outputs.
How was the propaganda evaluation designed?
Nous built an 803-prompt evaluation set spanning the seven categories, pulling prompts from English-language government statutes, news reports, and research reports. No single source contributed more than 13 prompts, which limits the risk of any one dataset or outlet skewing the results.
For every prompt, the team extracted two to four required facts from a dated source article, along with the specific official-frame language the Chinese state uses on that topic. Grok 4.7 served as an LLM judge, scoring each answer against the prompt, the required facts, and a fixed rubric, without being told which model produced the response.
The judge wasn’t just deployed and trusted blindly. Nous ran a 25-prompt pilot, hand-graded it, and revised the rubric three times after disagreements between human and judge grading. The key fix was scoring a “stated fact” by its substance rather than requiring an exact match on wording, dates, or citations.
Two separate scores came out of this process. The factual recall score (0 to 5) measures how many required facts the answer actually states, with a single “opposed” fact (where the model denies a fact or substitutes the state’s account) capping the score at 0 regardless of other performance. The propaganda score (0 to 2) measures whether the answer adopts the state’s framing, either replacing facts outright or endorsing state conduct alongside correct facts. Simply quoting the official position to identify it as such (“the Party describes this as…”) doesn’t count as propaganda under this rubric.
What were the benchmark results?
Across five runs over all 803 prompts with a 32k-token cap, Arete-27B scored a fact mean of 2.98 (± 0.03) with a 0.9% propaganda rate, running with thinking off at temperature 0. The base Qwen3.8-27B model, under the same settings, scored 2.32 (± 0.03) with a 22% propaganda rate.
For context, Nous ran the same evaluation against several other open and closed models at temperature 0.6: Inkling (the teacher model) scored 3.96 with 0.3% propaganda, Mistral Large 4 scored 3.65 with 0.3%, and GLM 5.3 scored 3.10 but with a 15% propaganda rate, while DeepSeek V4.1 Flash scored 2.39 with 16% propaganda. These comparisons used different decoding settings than the Arete-27B/Qwen head-to-head, so they’re directional rather than strictly apples-to-apples. When Nous reran Arete-27B and Qwen3.8-27B at the same temperature 0.6 setting, Arete-27B scored 2.88 and Qwen3.8-27B scored 2.24, confirming the gap held across sampling conditions.
The number of zero-scoring answers (meaning no required fact stated, or at least one fact opposed) fell from 359 to 216 out of 803 prompts. The improvement held across all seven categories, with the largest shift in freedom of expression: fact score rose from 0.89 to 2.44, and the propaganda rate dropped from 60% to 1%.
On general capability, Hermes Bench (Nous’s internal benchmark) showed Arete-27B at 65.60% versus Qwen3.8-27B’s 65.77%, a gap the team calls noise. On Terminal-Bench-2.1, an agentic benchmark, Arete-27B scored 70% versus the base model’s 67%, though Nous is cautious about over-crediting that improvement given differences in inference configuration and harness setup.
Is Arete-27B worth using?
For developers who want a Qwen-based model without state-aligned framing on politically sensitive topics, Arete-27B offers a documented, benchmarked alternative with no apparent capability tax based on the benchmarks Nous ran. The fact that Nous used supervised fine-tuning only, avoided reinforcement learning or architecture changes, and published both the model and the evaluation dataset gives outside researchers a way to independently verify the claims rather than taking them on faith.
- Name
- Maria Alvarez
- Date of birth
- SSN
- Diagnosis
- Balance due
- $412.80
That said, Nous is explicit that this doesn’t “remedy all behaviors associated with the extant viewpoint.” It addresses what the team identifies as a likely root cause within the seven categories tested, using a judge-scored methodology that, while rigorously piloted, still relies on another LLM’s judgment calls. Anyone evaluating the model for production use should treat the released Propaganda_Eval dataset as a starting point for their own testing rather than a final verdict.
Frequently Asked Questions
What base model does Arete-27B use?
Arete-27B is a fine-tune of Alibaba’s Qwen3.8-27B, a 27-billion-parameter model. Nous Research kept the architecture unchanged and trained all language-model weights using supervised fine-tuning.
What teacher model did Nous Research use for training data?
Nous used an open model called Inkling as the teacher. They selected it after benchmarking several open models against their internal evaluation criteria, and chose an open model specifically to sidestep “illicit distillation” restrictions that come with some closed-model terms of use.
How much did propaganda framing actually decrease?
On Nous’s 803-prompt evaluation, the share of answers containing propaganda-style framing dropped from 22% in the base Qwen3.8-27B to 0.9% in Arete-27B, measured with thinking off at temperature 0. At temperature 0.6, the gap was similar: 22.4% versus a lower rate for Arete-27B derived from its 2.88 fact score run.
Did fine-tuning hurt the model’s general capabilities?
Based on the benchmarks Nous ran, no. Hermes Bench scores were within 0.2 percentage points of the base model, and Terminal-Bench-2.1 scores were slightly higher for Arete-27B, though Nous attributes that difference partly to inference configuration rather than claiming a genuine capability gain.
Can other researchers verify these results?
Yes. Nous published the Propaganda_Eval dataset, containing all 803 prompts and their source articles, on Hugging Face, along with the Arete-27B model weights, so outside teams can rerun the evaluation against this model or any other.
