In Anthropic’s randomized trial, 50 professional analysts using Claude Code followed the option their agent had been secretly assigned by coin flip: on each of four analytical decisions, the share choosing option A was 62 to 82 percentage points higher when the agent leaned A than when it leaned B, while the final recommendation moved only 17 points (54% versus 36%). The gap matters because a human sign-off is supposed to be an independent check on AI-assisted work — yet analysts’ choices largely tracked a randomly assigned preference, and 77% later repeated the agent’s arguments while only 15% credited it. The finding comes from the analyst-deference study in the October 28 SI News roundup.
Sources & Notes
- Original story: Google DeepMind unveils Genie 4 world simulator | Anthropic finds analysts defer to AI agents | Malaysia seizes $12 million in AI chips
- Data sources: Herdağdelen, Joren, & Tamkin (2026). How AI agents shape experts’ judgment calls
- Notes: Shares are Figure 2 values from the Anthropic study: the share choosing option A when the agent leaned B versus leaned A, for 50 analysts (11 of 61 recruits were excluded because they chose not to use the agent). The agent’s lean was set by coin flip and deliberately strengthened so its effect could be measured; the study covers one task, one field, and one model (Claude Opus 4.8). The dashed diagonal marks the no-influence line where the lean would make no difference; the final recommendation counts a one-year extension as not rolling out the program. Legend labels shorten the five decisions: outcome measure, comparison method, farms covered, weighting, and final recommendation.
Leave a comment