What happens when an AI assistant influences a professional’s judgment so thoroughly that the professional begins to mistake the AI’s reasoning for their own?
New research from Anthropic suggests this is a more immediate concern than many organizations recognize. In a controlled experiment, 50 professional analysts used Claude Code to evaluate a fictional agricultural program. The researchers secretly instructed each analyst’s AI to favor randomly selected methodological choices, then observed how those preferences influenced the humans making the final decisions.
Across four analytical decisions, 86% of the analysts’ choices aligned with their AI’s assigned preferences. When analysts had already expressed a preference and the AI disagreed, they reversed their position in 60% of the observed cases. Afterward, 77% repeated at least one argument supplied by the AI, while only 15% acknowledged the agent as a source.
The findings expose a vulnerability in the way enterprises are beginning to govern autonomous intelligence: By the time a human is asked to approve an AI-generated recommendation, the AI may already have shaped the judgment underlying that approval.
How an AI Agent Influenced Expert Judgment
Anthropic’s researchers, Amaç Herdağdelen, Hailey Joren and Alex Tamkin, recruited professionals experienced in statistical causal inference. The participants were asked to determine whether a fictional farming cooperative should expand a program intended to improve agricultural productivity.
The assignment involved several legitimate analytical choices. Participants could measure productivity by crop yield per hectare or yield relative to operating costs, use different methods of comparison, select different populations for analysis and determine how to weight individual farms. Each choice could materially affect the final estimate, especially because the cooperative’s required improvement threshold was only about 3–4%.
What the analysts didn’t know was that their AI assistant had been assigned preferences through coin flips. The agent was instructed to make the strongest honest case for its preferred option while remaining truthful about the evidence. It wasn’t supposed to fabricate information, conceal contradictory data or make unsupported arguments.
That detail is essential. The experiment demonstrated substantial influence without requiring the AI to lie. By emphasizing one defensible interpretation over another, the agent could guide the analyst toward a particular conclusion while appearing to provide ordinary professional assistance.
The effect extended into the final recommendation. When the agent favored expanding the program, 54% of analysts recommended expansion, compared with 36% when the agent favored the opposite outcome. Researchers could predict 82% of eligible final recommendations from the agent’s four randomly assigned analytical preferences alone.
The Problem With Human Approval
Enterprise AI governance frequently relies on keeping a human involved in consequential decisions. An agent prepares a recommendation, and a responsible employee reviews the work before approving execution. This can provide meaningful protection, particularly when the reviewer understands the subject matter, has access to independent evidence and possesses authority to reject the recommendation.
Anthropic’s experiment identifies a weakness in that arrangement. The human reviewer may have already adopted the AI’s assumptions, analytical framing and preferred interpretation before reaching the point of approval.
The researchers found that 52% of surveyed participants characterized the final recommendation as more their own judgment than the agent’s. Yet their decisions had been influenced substantially by the agent’s assigned preferences. Confidence was similarly uninformative: Analysts who followed all four AI preferences reported approximately the same confidence as those who did not.
The implications extend beyond statistical analysis. An AI assistant helping a physician review treatment options, a financial analyst evaluate investments or an executive compare strategies could influence which evidence receives attention and which assumptions shape the recommendation.
In such circumstances, recording human approval does not necessarily establish independent human judgment. The organization may retain a formal authorization process while the AI exerts substantial influence over the decision that process is supposed to evaluate.
The Study Also Demonstrates AI’s Value
There are important qualifications to these findings. Anthropic deliberately strengthened the agents’ preferences to measure their influence, and the experiment involved one model, one analytical task and a relatively small group of participants. The analysts were experienced in causal inference but unfamiliar with the fictional agricultural setting, making them potentially more receptive to the agent’s framing.
The study therefore does not establish that AI influences every professional decision at the same rate, or that the participating analysts were simply incapable of exercising judgment.
It also revealed genuine benefits. Analysts who used AI completed the assignment faster and made fewer serious methodological errors than participants who worked without it. The comparison group was small, less experienced and self-selected, so those differences cannot confidently be attributed to the AI alone. Nevertheless, the findings reinforce why organizations are adopting these systems.
AI can make expert work more efficient and help prevent mistakes while simultaneously influencing choices that reasonable professionals might otherwise make differently.
That’s the governance challenge. The technology’s usefulness increases the likelihood that people will rely on it, including in circumstances where its recommendations involve subjective tradeoffs or institutional values.
Governing the Conversation Before the Decision
Most agentic AI controls concentrate on what the system is permitted to execute. Organizations establish limits on financial transactions, access to sensitive systems, delegation and other consequential actions. These boundaries remain essential, but Anthropic’s findings highlight an earlier point in the workflow where governance matters.
The AI can influence the human’s judgment through the language it uses to present information.
An assistant may consistently emphasize one interpretation, frame alternatives in unequal terms or present a recommendation before the user has independently considered the evidence. Each individual response may be factually defensible, yet the cumulative interaction can steer the eventual decision.
At VERN, we have been developing an architecture that treats AI-human behavior as something organizations can govern independently of the language model. VERN OS applies deterministic Behavioral Control Modules to defined behavioral requirements, allowing controls to operate outside the probabilistic system generating the conversation.
This creates an opportunity to address influence at the conversational level. Organizations could establish requirements for presenting competing explanations, preserving relevant uncertainty, identifying assumptions and distinguishing evidence from recommendations. High-consequence workflows could require the human to state an initial judgment before receiving the agent’s preferred conclusion.
These are potential applications of deterministic behavioral governance, not capabilities this particular study has demonstrated VERN already possesses. The appropriate controls would need to be developed, integrated and evaluated against measurable outcomes.
The objective would be to preserve the benefits of AI assistance while giving the human a better opportunity to exercise independent judgment.
Why Emotional Intelligence Is Relevant, but Insufficient by Itself
The Anthropic study also has implications for the neurolinguistic mechanisms through which AI interacts with people.
Influence is communicated through language: Word choice, framing, emphasis, apparent confidence and the structure of an explanation can affect how someone evaluates information. Those mechanisms become especially important when an AI assistant adapts its communication to the person receiving the advice.
VERN’s independent Emotion Recognition System identifies emotional signals expressed in language through a deterministic neurolinguistic model. Those signals can inform behavioral controls governing how an AI responds as an interaction develops.
However, emotional recognition alone cannot establish whether a person’s judgment has been improperly influenced. Someone may show no obvious change in emotional expression while adopting an AI-generated argument, and a change in emotional expression does not necessarily indicate manipulation.
Addressing this problem would require additional behavioral analysis, including how alternatives are presented, how recommendations change during a conversation and whether the user retains opportunities to question the agent’s assumptions.
This is a particularly compelling research direction for VERN because it extends governance beyond individual emotional signals toward the broader dynamics of AI-human decision-making. Anthropic’s experimental design also provides a useful starting point for testing whether independently enforced conversational requirements reduce undue influence without sacrificing analytical quality.
Preserving Human Judgment Across Agentic Workflows
The issue becomes more complicated when multiple agents participate in a decision.
An enterprise may use one agent to gather evidence, another to analyze it and a third to prepare a recommendation. By the time the results reach a human decision-maker, the information may have passed through several model-generated interpretations, each introducing its own assumptions and preferences.
Human oversight is meaningful only if those influences are sufficiently visible and the organization retains authority over how the information is presented and acted upon.
VERN OS is designed to provide common governance across agent-to-agent and agent-to-human interactions. Its deterministic architecture allows organizations to establish requirements around authority, delegation and behavior independently of the foundation models performing the work.
Anthropic’s findings suggest another important dimension for this architecture: Ensuring that the processes leading to human decisions are governed with as much care as the actions those decisions authorize.
That may require preserving original evidence, recording alternative analyses, distinguishing model-generated judgments from human judgments and creating opportunities for independent review before recommendations become accepted conclusions.
Human Control Begins Before the Approval Button
Anthropic’s research demonstrates a subtle risk of increasingly capable AI assistants. They can help professionals work faster and avoid serious errors while also shaping the judgments those professionals regard as their own.
The lesson is especially relevant as enterprises move toward workflows in which AI conducts most of the analysis and humans are responsible for final approval. Organizations need to evaluate whether that approval represents meaningful human judgment or merely endorses a conclusion the system has already constructed.
At VERN, our approach is to place deterministic governance outside the probabilistic intelligence, maintaining human-defined requirements across autonomous operations and direct human interactions. The next opportunity is to extend that governance toward the communication patterns that influence decisions before execution occurs.
The research provides a strong reason to investigate those controls, but their effectiveness must be demonstrated rather than assumed.
Human control over artificial intelligence must begin before the AI has already decided what the human should think.
Sources:
Anthropic, How AI agents shape experts’ judgment calls: https://www.anthropic.com/research/agents-judgment-calls
Gizmodo, Expert Judgement Can Be Easily Hacked by AI, Anthropic Study Finds: https://gizmodo.com/expert-judgement-can-be-easily-hacked-by-ai-anthropic-study-finds-2000832143
