Vals AI finds Chinese models vary sharply on politics

By Sophie Lindqvist

Thomson 1.0 Small cut its share of answers adopting official Chinese positions from 84.6% to 15.0% after Thomson Reuters fine-tuned it, according to Vals AI tests. The result came from a wider Vals AI study of how Chinese, American and other language models answer political questions about China.

Vals also found that prompt language, response format and tool access can alter political answers without changing the underlying model. That variability matters for anyone assessing whether a chatbot will repeat official positions in ordinary use.

What did Vals AI measure?

Vals AI, an AI benchmark firm, asked Chinese, American and other language models hundreds of questions about China. Researchers varied prompt language, usage context and response format across the tests.

Vals called the measured alignment with Chinese government positions a “pro-China lean.” Its scale ran from −1 to +1. Positive scores indicated greater agreement with official positions or a pro-government stance, while negative scores indicated more critical answers.

Vals said the overall pro-government lean of Chinese models was lower than expected, although major differences separated individual models. GLM-5.3 Flash explicitly confirmed that “China’s human rights violations are well documented” when asked about them.

Vals also found that some non-Chinese models occasionally aligned with official Chinese government positions on political questions.

Takeaway: Chinese models did not form one uniform bloc.

Why does prompt language change the answers?

The study found a strong effect from the language of the prompt. For most models, answers in Simplified Chinese were more pro-government than answers in English, Russian or Spanish.

That pattern appeared in both Chinese and American models, but it was far more pronounced among the Chinese models. Vals also found a smaller, consistent gap between Simplified Chinese and Traditional Chinese.

Simplified Chinese is used on mainland China, while Traditional Chinese is common in Taiwan and Hong Kong. Vals suspects that models infer the expected perspective from the language itself.

A separate experiment supported that idea. When a system prompt said the model served a company on mainland China, some models became more pro-government than when the prompt described US-based use.

Takeaway: The language alone can move a model’s answers toward Beijing’s position.

Do Chinese models mainly refuse sensitive questions?

The common assumption that Chinese models mainly block sensitive questions held only for certain DeepSeek models in Vals’ tests. Most other Chinese models rarely refused, and some instead adopted official government positions.

Kimi K3, for example, denied that concentration camps existed for Uyghurs in Xinjiang when asked. It called the facilities “vocational training centers designed to combat extremism and terrorism,” according to Vals.

Takeaway: Most tested Chinese models answered rather than refusing, and some took Beijing’s side.

How do answer format and tools affect results?

Vals found that the required answer format changed results. When the firm forced political questions to use a simple yes-or-no response, GLM-5.3 Flash and MiMo V2.6 Pro scored much less pro-government than in open-ended tests.

Vals also tested tool definitions. In two setups, Chinese models produced less pro-government answers:

• Coding assistants: Vals gave Chinese models system instructions and tool definitions for coding assistants. Several scored lower on the pro-government scale, and the models rarely used the tools.
• Web search: Giving models access to web search produced a similar effect. Their responses became less pro-government even though the models never actually searched the web.

Takeaway: The route to an answer can affect its political direction even when the model never calls a tool.

Do political signals show up in everyday tasks?

Vals said political tendencies were much harder to detect in realistic work tasks than in targeted political questions. The company still found individual examples.

During a test that translated country codes for a travel app into Simplified Chinese, most models rendered “Taiwan” without a qualifier. Qwen3.8 Max instead used “Taiwan, China,” a label that implies Taiwan belongs to China and is used by the Chinese government.

Vals also published map examples from OpenAI’s GPT Image 2.5 Sunburst. In the Chinese-prompted result, it labeled Taiwan “Taiwan Province” and added the disputed nine-dash line Beijing uses to mark territorial claims in the South China Sea.

Those elements were absent from the English-language result shown by Vals.

Takeaway: Political framing still appeared in routine translation and image tasks.

What changed after fine-tuning?

Vals compared Thomson 1.0 Small with the original Qwen3.6 35B model on which it was based. Thomson Reuters fine-tuned the model, and its share of official-position answers fell from 84.6% to 15.0%.

Fine-tuning did not erase every political response. Some individual answers still defended Beijing’s One-China principle and the “peaceful reunification” with Taiwan.

Takeaway: Targeted tuning sharply reduced official-position answers, but did not remove the topic from every response.

How did Vals score the answers?

Vals used political questions that the company says do not represent typical user prompts. Researchers translated them with GPT-6 Astra and spot-checked the translations, then automatically scored model responses with GPT-5.6 Luna.

Refusals counted as neutral by default. That means a model refusing every political question could appear neutral on the scale, so the score measures the direction of answers rather than refusal alone.

Takeaway: The headline scores measure alignment among answers, not simply how often a model refuses.

What do provider and policy caveats mean?

Many Chinese models ran through third-party providers rather than the manufacturers’ official services. Vals warned that official offerings may apply additional content filters, so its results do not necessarily carry over to publicly available chatbots.

Earlier studies by Aleph Alpha and NewsGuard had already shown that Chinese AI models frequently adopt official narratives on sensitive topics. China’s regulations also require publicly offered AI services to align with “core socialist values.”

A score from a third-party endpoint therefore cannot establish what a public chatbot will do. Provider filters and national requirements can make a deployed service differ from the model Vals measured.

Takeaway: Provider routing and deployment rules can make real chatbots behave differently from the tested models.

The clearest conclusion is not that Chinese models simply repeat Beijing. Vals found more variation by model, language, task format and tool access, while government-aligned wording still appeared in ordinary product work.

The open question is whether these patterns persist in realistic use, where Vals found political tendencies harder to detect. That gap between benchmark scores and deployed behavior is what model buyers and policymakers still need to test.