Experts follow an AI agent’s random advice and think it’s their own

By Sophie Lindqvist

Anthropic’s study found that 50 experts followed an AI agent’s coin-assigned preference in 86% of four analytical decisions, while only 15% mentioned AI in later explanations.

That matters because the intermediate choices shaped a final policy recommendation. The study also recorded faster work and fewer serious flaws with AI, though a small comparison group makes the speed and quality gains hard to isolate.

Two stakes follow from the result. Research teams using the same model could create false consensus, while safety reviewers could lose the independent judgment meant to catch a misaligned system.

What did Anthropic’s experiment measure?

Anthropic’s study placed 50 experts in causal inference on a realistic analysis task. They examined observational data from a fictional farming cooperative and estimated the productivity gains from a voluntary farm advisory program. Participants then had to recommend ending the program, extending it for one year, or expanding it to all farms.

The study contained four analytical choices that shaped the work, including how to measure productivity and which statistical method to use. Anthropic designed the questions without a clearly right answer.

A coin toss set the agent’s preference between two valid options at each decision point. The agent could not lie or hide data. It only had to make the strongest honest case for its assigned side.

Participants adopted the agent’s preference in 86% of the four analytical decisions, far above the 50% expected from random choice. Anthropic’s researchers called the agent’s net influence the “flip rate,” which ranged from 62% to 82% depending on the task.

Anthropic’s team frames the result as a question about how people divide work with machines:

"There may be no single right division of labor between people and AI. Some decisions may be fine to hand over, and others worth keeping. With the right tools, that split can be something we choose, instead of something we drift into," the researchers write.

The agent’s assigned preference had only a small direct effect on the final policy recommendation to the cooperative’s board. It slightly shifted votes without reaching statistical significance, and some experts overruled the agent when its advice did not match their calculated estimate of productivity.

That apparent final check did not erase the earlier influence. Anthropic’s researchers say the four analytical decisions had a major effect on estimated productivity gains, which in turn shaped the recommendation.

The agent could shape the assumptions and methods behind a recommendation that experts still presented as their own.

Did expertise protect participants?

Having an opinion before the agent spoke offered little protection. When the agent advocated the opposing view, participants were 53 percentage points more likely to switch than when it agreed with them.

The researchers tested years of analysis experience, self-rated expertise, education, test scores, and trust in AI. None showed a statistically reliable link to the results after correcting for multiple tests.

Age stood out before that correction. The researchers cautioned that the sample of 50 gave the study little power to detect age effects.

Expertise, education, and trust did not reliably protect experts from the agent’s framing.

Why did experts call the result their own?

After the task, 52% of participants said the recommendation felt more like their own than the agent’s. Another 35% said it was mostly or entirely their own.

Participants correctly recalled their own choice for 85% of decisions. Yet 77%, or 37 of 48, used at least one of the agent’s arguments in their explanations, sometimes in nearly the same words. Only 15% mentioned AI at all.

The conversation logs showed the agent stated a position first in 77% of decisions. It usually argued only for its preferred side and did not explain why skilled experts might choose the alternative. It did explain how a different choice would affect the result.

A final recommendation can look independent even when the reasoning behind it has already absorbed the agent’s framing.

Did AI improve the work?

The study found that participants completed the task faster and made fewer serious errors with the agent. Without AI, 55% of analyses, or 6 of 11, had serious methodological or coding flaws. The median completion time was 90 minutes.

With AI, just 6%, or 3 of 50 analyses, had serious flaws. The median completion time fell to 65 minutes.

The no-AI group was small and had chosen not to use the tool. It also had somewhat less expertise, so the researchers said they could not determine how much of the gap came directly from AI.

The reported gains in speed and quality came with a quieter cost: participants often failed to identify the agent’s role in their own reasoning.

What could shared AI preferences change?

The researchers warn that quiet deference could have broad effects in science. If many teams use the same model with fixed default preferences, they could end up creating a false consensus.

That risk extends to AI safety itself. Researchers already use AI agents to test frontier models for safety and alignment, expecting human review to serve as an independent check. If experts unknowingly adopt an agent’s bias on open-ended judgment calls, a misaligned system could steer the research meant to expose that misalignment.

The team calls for ways to measure deference to AI agents in real work. It suggests that agents should make the best case for both sides, or ask experts for their own view before stating a position.

Institutions also need to identify which calls require independent human judgment. They should set aside time and resources for those decisions rather than treating review as automatic.

Shared defaults can make independent review look like agreement, even when people have not consciously chosen the same answer.

What does earlier research show?

Earlier studies cited alongside Anthropic’s experiment point in the same direction, though the evidence remains early and mixed.

• A false-confidence study with 3,132 participants found that seeing AI answers nearly erased people’s willingness to withhold judgment. In one part, the share willing to say “I don’t know” fell from 44% to 3%.
• A cited study of polished AI outputs, such as code or documents, found that users checked facts 3.7 percentage points less often, questioned its reasoning 3.1 points less often, and missed missing context 5.2 points more often. Users who revised outputs step by step questioned the AI’s reasoning 5.6 times more often than those who did not iterate.
• A Polish observational study of colonoscopies, published in The Lancet Gastroenterology & Hepatology, found that after experienced doctors regularly used an AI system, their rate of finding precancerous lesions in exams without AI fell from 28.4% to 22.4%. The study’s authors suggested that routine AI use may affect people’s own cognitive skills.
• A Swiss Business School survey found a strong negative link between AI tool use and critical thinking, mainly among people ages 17 to 25. A solid education appeared to offer some protection, as people with higher degrees questioned AI answers more and engaged more with complex thought processes.
• A Nigerian pilot project pointed in the other direction. Students who used a chatbot as a virtual tutor twice a week made learning gains in six weeks equal to nearly two extra years of learning.

Anthropic’s result remains a warning about defaults and delegation. Teams will need to test whether an apparent consensus came from independent judgment or a shared system.