Researchers removed one instruction from Claude Code. Out-of-scope actions jumped.
Anthropic is reportedly developing a model that can infer what users want from incomplete instructions. According to The Information, a person who tested the unreleased model described stronger reasoning and intuition than the company’s current lineup.
In one example, a developer asks the model to fix a bug. Instead of fixing that one problem and moving on, the model spots similar issues elsewhere in the code and asks whether the developer wants those fixed, too.
Anthropic hasn’t confirmed the model or said when it might be released. But the reported capability raises a question for developers: How much freedom should a coding agent have to decide what needs fixing?
When coding agents go too far
A May study, “Overeager Coding Agents,” looked at roughly 7,500 runs involving four coding-agent products and six models. In a paired experiment with 76 scenarios, researchers removed a prompt section that explicitly defined authorized and out-of-scope actions. With Claude Code running GLM-4.6, the rate of out-of-scope actions rose from 0% to 17.1%. With Claude Code running Anthropic’s Sonnet 4.6, it rose from 3.9% to 15.8%.
The researchers ran Claude Code with its --dangerously-skip-permissions setting, which disables permission prompts. The researchers kept those settings the same across the paired tests and changed only the prompt’s scope declaration. The results therefore measure behavior with permission checks disabled, rather than performance with approval safeguards enabled.
The findings apply to the models and configurations tested; they do not establish failure rates for current Claude Code configurations. The study did not test the unreleased model described in The Information report.
Anthropic also offers auto mode, which uses automated checks designed to block risky actions beyond the user’s intent. The company publicly described the feature in March, before the May study. The researchers did not evaluate auto mode, and Anthropic acknowledges that its checks can miss unauthorized actions.
A separate July study, “Coding Agents Are Guessing,” tested Claude Code, Codex, and OpenCode on DevOps tasks with incomplete instructions. All runs used fully autonomous execution settings. Across five agent-model configurations, 55.8% to 67.8% of runs in which agents took action crossed at least one operational boundary.
An agent might be smart enough to spot problems a developer missed, but that doesn’t necessarily mean it has permission to fix them.
The findings highlight a real challenge for coding agents. An agent might be smart enough to spot problems a developer missed, but that doesn’t necessarily mean it has permission to fix them.
Overeager agents by the numbers
The May study also found substantial differences between agent frameworks under the tested configurations. Claude Code, Codex CLI, and Gemini CLI recorded overeager-action rates ranging from 5.4% to 27.7%, compared with 0.2% to 4.5% for OpenHands. Even within the same framework, changing the underlying model produced differences of up to 15.9 percentage points.
The Coding Agents Are Guessing study tested the agents against 2,208 underspecified DevOps prompts covering 69 task families.
The researchers distinguished between agents acting on the wrong target and agents going beyond the permitted scope. Both are problems, but they aren’t interchangeable since an agent can perform exactly the requested operation and still cause trouble by choosing the wrong resource.
When the target is unclear
In these tests, unclear targets were among the biggest problems. Warnings about the potential impact of an action did little to reduce how often agents proceeded anyway.
Warnings about the potential impact of an action did little to reduce how often agents proceeded anyway.
Both studies raise questions about whether improvements in reasoning will also improve an agent’s judgment when instructions leave room for interpretation.
Keeping agents within bounds
Anthropic has encountered a related problem in its own testing. In an October 9 report on unintended model actions, the company described cases where Claude kept looking for ways to finish an assignment after encountering a restriction.
Most of the cases involved what Anthropic called persistence, where Claude worked around an obstacle rather than stopping.
In one evaluation, Claude Haiku 4.5 was told to complete a form but stop before submitting it. The model submitted it anyway, apparently expecting another confirmation page. In another, an unreleased research model submitted a real government form after the practice version failed to load or was accidentally closed.
While these were not coding tasks, and the company has said the incidents had minimal real-world impact, they still show how an agent can mistake a restriction for a problem it needs to work around.
Anthropic has acknowledged that clearer instructions might have prevented some of the failures.
Anthropic has acknowledged that clearer instructions might have prevented some of the failures. The company also said it had developed monitoring tools that blocked every documented case when tested against them.
What the code cannot reveal
Anthropic will need to show that its model can tell the difference between an incomplete request and an intentional constraint. A developer might limit a fix because of an upcoming migration, compatibility requirements or a production freeze — none of which would necessarily be visible in the code.
Update: After publication, Anthropic responded to The New Stack’s request for comment. This article has been updated to clarify the models and testing conditions and to include information about Claude Code’s auto mode.
