Thinkr

Trends

Developers Felt 20% Faster. They Were 19% Slower.

A controlled trial found developers slowed down with AI while believing they had sped up. The cost did not vanish — it moved somewhere nobody measures.

Galang Aulia · 4 min read
Trends

Two pieces of research from the last eighteen months point at the same thing from opposite directions, and together they describe a problem most teams are currently having without a name for it.

What the trial found

In July 2025, METR published a randomized controlled trial of AI-assisted development. Sixteen experienced open-source developers completed 246 real tasks on repositories they already knew well, randomly assigned to allow or disallow AI tooling.

Before starting, the developers expected AI to make them 24% faster. Afterwards, they estimated it had made them 20% faster. Measured, they were 19% slower.

The gap between the second and third numbers is the interesting one. This was not a case of people predicting badly in advance — that is ordinary. These developers had just done the work, and still could not tell which way it had gone.

The authors are careful about what this does and does not show, and it is worth repeating their caveat rather than burying it: the study does not demonstrate that AI fails to speed up most developers. It measured a specific population — experienced contributors, on codebases they knew deeply, with early-2025 tools and roughly fifty hours of exposure each. Sampling bias is plausible; developers who had found AI transformative may simply have been less likely to enrol. Someone learning an unfamiliar codebase, or writing something routine, may well be faster.

Take it as a narrow, well-measured result rather than a verdict.

What the survey found

DORA's March 2026 analysis comes at it from the other end: not a trial, but adoption at scale. 90% of technology professionals now use AI at work, and over 80% believe it has made them more productive.

The delivery data complicates that belief. Higher AI adoption correlates with an increase in software delivery throughput and an increase in software delivery instability — more change failures, more rework, longer recovery. Both at once, in the same organisations.

DORA's explanation is the useful part, and they give it a name: the verification tax. Time saved generating code gets reallocated to auditing it. The constraint does not vanish when generation gets cheap; it relocates to whatever now has to absorb the increased volume. And review capacity does not scale just because generation did.

A related figure from the same work: 30% of developers report little to no trust in the code AI generates. They are already paying the tax. They simply have not been given anywhere to put it on the ledger.

Why the two results are the same result

Generation is legible and verification is not.

You can watch a working implementation appear. That registers as progress, immediately and viscerally. Re-reading it to find the thing it got quietly wrong does not register as anything — it feels like overhead, it produces no artifact, and when it succeeds the outcome is that nothing happens.

So the time moves from a stage people notice into a stage people do not, and the self-report follows the visible half. Twenty percent faster is an honest description of the part of the work that is easy to see.

This is also why the instability shows up downstream rather than at the moment of writing. A model handed an underspecified task does not object. It produces a confident, plausible implementation of one interpretation, and the interpretation is invisible unless somebody goes looking. The ambiguity does not surface as an error. It surfaces later as a finished feature that is subtly not what anyone meant, which is a change failure in the metrics and a rewrite in the calendar.

What follows for the people writing the specs

If verification is where the cost went, then the quality of what gets verified against is now doing more work than it used to.

An ambiguous spec is more expensive than it was two years ago, not less. When building was slow, the slowness was a buffer — a week of implementation left room for someone to notice the gap and ask. Compress that to an afternoon and the buffer is gone. The question that would have surfaced mid-build now surfaces in review, after the thing exists, and the second build costs what the first one did.

Fluency stopped being evidence. Any model produces a well-structured document on request, which means structure no longer signals that anyone thought carefully. A spec can be correctly sectioned, confident throughout, and contain no decision at all — and it will pass a skim, because skimming checks for fluency and fluency is now free.

The scarce skill is reading, not writing. Everything above describes a verification problem, and verification is the thing almost nobody was trained to do. People were taught to write documents. Being handed a competent-sounding one and locating the decision that is missing from it is a different muscle, and it is now sitting on the critical path of everything a team ships.

The tooling caught up to the writing. It has not caught up to the judgment, and the measurement problem is the reason: you cannot feel the stage you are losing time to. That is precisely the kind of cost that goes unmanaged, because managing it starts with being able to see it.

Sources

  1. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer ProductivityMETR ·
  2. Balancing AI tensions: Moving from AI adoption to effective SDLC useDORA (Google Cloud) ·
New posts and release notes. No spam, unsubscribe anytime.

Stop shipping foggy PRDs.
Start the critique loop.

Three minutes to sign up. No credit card. Cancel by closing the tab.

Start freeSee pricing