Nous says fine-tuning cuts Qwen3.8-27B propaganda to 0.9%

Nous Research says supervised fine-tuning (continued training on curated examples) on 53 million training tokens (units of text) cut the propaganda rate of Alibaba’s Qwen3.8-27B from 22% to 0.9% on an 803-prompt test, while factual accuracy rose from 2.32 to 2.98 out of 5.

The US startup, best known for its open-source agent Hermes, calls the resulting open-weight model (one with downloadable trained parameters) Arete-27B. Nous released the weights, test dataset and training data under an Apache 2.0 license on Hugging Face.

According to Nous, Chinese open-weight models rank among the strongest available but carry state-mandated behavior baked into their weights. The team observed refusals, evasive answers and responses aligned with the Chinese Communist Party in Qwen3.8-27B.

Every headline result comes from Nous’s own evaluation, and the source report contains no independent replication of the 0.9% figure. That leaves the headline dependent on Nous’s prompts, reference facts, judge model and scoring definition rather than independent validation.

What does the 803-prompt test measure?

The test used 803 prompts across seven categories: free speech, state ideology, education, privacy, surveillance, democracy and the environment. For each prompt, Nous defined two to four facts drawn from a dated article, such as a law, news report or research paper.

A response counted as propaganda if it reproduced Beijing’s official narrative instead of the facts, endorsed that narrative or treated it as an equally valid perspective. That definition can include cautious framing as well as explicit endorsement, so the headline rate depends on a contestable boundary.

Grok 4.7 from Elon Musk’s xAI served as the judge, checking whether an answer mentioned, omitted or contradicted the reference facts. Nous describes the judge as politically right-leaning. The judge had to cite the response’s specific wording and did not know which model produced the answer.

The results were:

• Unmodified Qwen3.8-27B: 2.32/5 for factual accuracy; 22% propaganda.
• Arete-27B: 2.98/5 for factual accuracy; 0.9% propaganda.
• Mistral Large 4 and Gemma 4 31B: below 1% propaganda each.
• GLM-5.3: 15% propaganda.
• DeepSeek V4.1 Flash: 16% propaganda.

The largest category change came in free speech, where the fact score rose from 0.89 to 2.44 and the propaganda rate fell from 60% to 1%. Arete with reasoning mode reached 3.07, but that result used only two of five runs completed so far.

Those comparisons show that the base model was not an isolated outlier within the tested set. Takeaway: The benchmark demonstrates a sharp drop under Nous’s scoring rules, but its design leaves room for subjective judgments.

How did fine-tuning change the model?

Nous chose Inkling from Thinking Machines Lab as its teacher model (the model that generates target responses) because an open model would not break terms of service that ban distillation (training a model from another model’s outputs). The team also shortlisted Nvidia’s Nemotron 3 Ultra but reports signs of irregularities on China-related topics.

Nous suspects the irregularities came from training on Chinese open-weight models. It reports similar contamination in its own projects, which narrows the pool of models the team treats as clean teacher candidates.

Thinking Machines used models including the Chinese Kimi K2.5 for Inkling’s synthetic training data (machine-generated examples). The Kimi detail matters because Inkling’s pipeline touched a Chinese model before Inkling trained Qwen. It does not prove Kimi’s behavior passed into Arete, but it weakens any assumption that the teacher chain was free of Chinese-model influence.

Nous trained Qwen3.8-27B with supervised fine-tuning on about 53 million tokens over two epochs (passes through the training data). The team used neither reinforcement learning (reward-driven training) nor preference optimization (training against preferred outputs), and says it removed overlaps between training and test prompts beforehand.

Nous writes, “that the bias layer in a Chinese-aligned model is shallow.” It adds, “It is applied late, it is applied thin, and fifty million tokens of carefully chosen supervision removes it without disturbing anything underneath.”

The claim remains narrower than its political framing. Evidence from one base model, one teacher and one benchmark cannot establish that Chinese open-weight models are uniformly superficial or that every capability remains unchanged.

The source does not report ablation studies (comparisons that remove one variable) across teachers, training mixes or base models. Takeaway: The result supports a removable behavior layer in this experiment, not a universal rule about every Chinese model.

Did Arete-27B retain its capabilities?

Nous says general capabilities did not suffer after fine-tuning. The company cites nearly identical results from its internal Hermes Bench, where Arete-27B scores 65.60% against 65.77% for the base model.

On Terminal-Bench 2.1, the score rises from 67% for the base model to 70% for Arete. Nous does not call that a real gain because the test setups differed, a caveat that makes the numerical comparison weak.

Together, the company reports little capability regression. The evidence is limited, however: Hermes Bench is internal, and the source provides no external audit of the model’s broader behavior.

Neither benchmark measures political neutrality after deployment or behavior under adversarial prompts. Takeaway: Nous reports little capability regression, but the benchmark design limits how strongly that promise generalizes.

Does the wider evidence support a China-versus-West divide?

NewsGuard found in 2025 that five leading models, including Qwen, repeated or failed to correct pro-Chinese disinformation in 60% of cases. The finding supports concern that political alignment can affect factual reliability, although its cases and method differ from Arete’s test.

CrowdStrike found that DeepSeek generated less secure code when queries referenced Falun Gong, Tibet or Taiwan. That result turns the debate from content preferences into a security question, because a model’s response to political triggers can change its code output.

An Aleph Alpha study used 967 politically sensitive questions to test models from Alibaba, DeepSeek and Moonshot AI. Only 17% to 41% of their answers were balanced, according to the study.

A recent ValsAI study showed that Chinese models do not behave uniformly. GLM-5.3 Flash explicitly confirmed human rights violations, while Kimi K3 denied the existence of internment camps.

ValsAI also found that language and response format clearly affected results, while Western models sometimes adopted official Beijing positions. The same study found targeted fine-tuning helpful, which supports the mechanism behind Arete without making all Chinese models alike.

The wider record complicates both claims of Chinese uniformity and Western immunity. Takeaway: Evidence supports targeted correction, but model origin alone does not determine political behavior.

How does Arete differ from Hermes 3?

With Hermes 3 in 2024, Nous deliberately chose neutrality and built the agent to adapt to whatever worldview appeared in its system prompt (runtime instructions). Arete takes a different path by embedding a political preference during training.

That shift turns alignment from a prompt-time choice into a durable property of the released weights. It also makes Arete’s political profile part of its product pitch rather than a neutral default.

The shift gives Nous a more marketable position in a market where some customers want models with explicit ideological preferences. It also leaves open whether political neutrality was more useful than a predictable stance for enterprise buyers.

That makes the benchmark a product claim as much as a research result. Takeaway: Arete breaks with Hermes 3’s neutrality, and its value proposition depends on users accepting that ideological choice.

What does the open release change?

Nous raised $90 million in a Series B round in October at a $1.5 billion valuation. Robot Ventures led the round,