Anthropic: Domain Expertise Matters More Than Coding Skills in Claude Code

Anthropic published a report based on an analysis of nearly 400,000 Claude Code sessions, showing that success in working with the coding agent depends mainly on the ability to precisely formulate tasks and evaluate results, not prior programming experience. The company tracked the behavior of about 235,000 users over seven months and divided them into novices and experts based on how they used the tool.

The report, titled "Agentic coding and persistent returns to expertise," is based on transcripts analyzed with privacy-preserving classifiers built on the Claude model. Its authors are Zoe Hitzig, Maxim Massenkoff, Eva Lyubich, Ryan Heller, and Peter McCrory from Anthropic's research team. Instead of asking how many lines of code a given user wrote, the researchers examined how precisely they could break a problem down into steps and judge whether the agent's answer was correct.

Novices versus experts

The difference between the two groups is visible already at the level of a single prompt. In a typical novice session, one prompt triggers about five Claude actions and generates roughly 600 words of output. In an expert session, the same prompt triggers twelve actions and more than 3,200 words, more than double the chain of actions and five times the output text.

This translates directly into effectiveness. Among novices, fully verified success was recorded in 15 percent of sessions, and any partial progress in 77 percent. Among experts, those figures rise to 28-33 percent and 91-92 percent, respectively. Novices abandon a session without any result almost four times more often than advanced users.

A shift in the nature of the work

The report's authors point to a clear evolution in how Claude Code was used over the study period. The share of sessions devoted to fixing bugs in code fell from 33 to 19 percent, while the share of tasks related to operating software, planning, and data analysis grew. In other words, the agent is increasingly used not just for firefighting but for broader organizational and decision-making tasks.

By task category, writing new code accounts for 25 percent of sessions, fixing bugs for 26 percent, testing and cleanup for 5 percent, and the remaining 44 percent covers operating software, planning, exploration, and data analysis and documentation. The report also notes that the average value of completed tasks rose 27 percent overall over time, and by as much as 43 percent for building new features.

Dividing roles between human and model

Anthropic also measured who makes which decisions during collaboration with the agent. The data show that users are responsible for about 70 percent of planning decisions, meaning determining what needs to be done, while Claude handles about 80 percent of execution decisions about how to carry out a given task. This division of labor matches the observation that the key human skill is becoming precisely defining the problem rather than manually writing every line of code.

A person with mastered domain knowledge in any field can now perform technical work they previously would not have been able to do - from the Anthropic report "Agentic coding and persistent returns to expertise"

Not just engineers

One of the more surprising findings of the report is the small difference in effectiveness between software engineers and people in other professions. Engineers achieve 30 percent verified success in coding sessions, while the other ten largest professional groups fall within 4-5 percentage points of that result. Among the professions increasing their use of Claude Code in agentic coding the fastest, Anthropic lists management, sales, and law, industries traditionally far removed from software engineering.

For Polish companies and IT teams, the report's findings carry practical weight for planning coding-agent rollouts. If effectiveness depends mainly on the ability to clearly describe a problem and verify the result rather than programming tenure itself, investing in training employees to formulate tasks well may yield a bigger return than simply giving the technical team access to the tool.

The report also shows that the benefits of agentic coding grow with the time spent using the tool, not just from one-time training. The 27 percent increase in the value of completed tasks over seven months suggests that organizations that have already adopted Claude Code are still learning to use it more effectively, rather than plateauing after the first few weeks.

Anthropic did not say whether it plans another edition of the study or whether a similar analysis will cover other tools in the Claude family. The company notes, however, that the data comes exclusively from interactive sessions conducted via the command line, the desktop app, and Claude.ai, so it does not cover the agent's use in automated pipelines without human involvement.