The incident had been running for four hours when the CTO joined the bridge call. Their lead engineer, the one who had debugged the original payment gateway integration eighteen months ago, the one who’d written the runbook everyone else follows, was still stuck. Not blocked in the usual sense. There were no missing permissions, no unfamiliar codebase, no dependency they hadn’t seen before. They were generating suggestions, applying them, watching the metrics, generating more suggestions. It looked like work. It produced nothing.
The CTO watched for a few minutes before asking: “What do you think is actually happening?” There was a pause. Not the confident pause of someone formulating a diagnosis. The other kind.
That call was three months ago. The CTO still hasn’t found a clean way to name what changed.
The Numbers Behind That Call
Two rigorous studies landed in 2025 and early 2026 that, taken together, explain a lot of what that CTO witnessed.
The first was a randomized controlled trial by METR, studying 16 experienced developers working on their own mature open-source projects. The developers predicted they’d be 24% faster with AI tools. After each task, they rated themselves 20% faster. Objective measurement showed they were 19% slower. That’s a 40-point gap between how the work felt and what it actually produced.
The second was a study from Anthropic, looking at 52 developers learning a new programming library. Those who used AI assistance scored 17 percentage points lower on comprehension and debugging quizzes than those who worked without it. Two letter grades. And the largest gap wasn’t in syntax or general recall. It was in debugging, the exact skill you need when production breaks.
These aren’t fringe findings. These are gold-standard RCTs with control groups. And the thing that makes them genuinely alarming for anyone running an engineering team isn’t the numbers themselves. It’s that the degradation is invisible to the people experiencing it.
If you feel faster while getting slower, you have no internal signal to course-correct. You increase AI reliance. You accelerate the underlying skill degradation. You feel increasingly confident throughout.
Why Debugging Is the Canary
Every production system, at some point, breaks in a way nobody anticipated. The runbook doesn’t cover it. The logs are ambiguous. The person on the call needs to reason from first principles about what could produce this specific combination of symptoms, form a hypothesis, and test it methodically. This is debugging. It is not glamorous. It is the single most important engineering skill that keeps businesses running overnight.
Here’s what AI assistance does to that skill: it interrupts the error-encounter-diagnose-resolve cycle that builds debugging intuition. When an AI suggests a fix and you apply it without working through why the error occurred, the error gets resolved and the learning doesn’t happen. The Anthropic study found that control-group participants encountered a median of three errors per session. AI-assisted participants encountered one. Those extra errors weren’t friction. They were training.
Over months of daily AI-assisted work, the gap compounds. Engineers who used to diagnose issues by pattern-matching against years of similar failures start reaching for the autocomplete instead. The pattern-matching library stops getting updated. And then there’s a 4am production incident and the bridge call goes quiet in exactly the way that CTO described.
The Organizational Blind Spot
Here’s why this problem doesn’t get caught before it matters: the metrics that engineering managers track are all pointing the right direction.
More PRs merged. More features shipped. More lines of code. These numbers all go up when you roll out AI coding tools. And they should, the tools do accelerate output, at least in the short term. The problem is that none of these metrics capture whether the team can reason independently about hard problems.
A 2026 analysis of AI adoption in engineering organizations found that managers with shorter time horizons consistently pushed for higher AI utilization rates than the engineers themselves who were thinking about their own technical development. The short-term productivity signal overrides the longer-term capability signal, and nobody puts “engineering team’s independent reasoning ability” on the quarterly OKR dashboard.
Ford ran into this at scale. After leaning heavily into AI-assisted engineering across several product lines, the company ended up rehiring engineers to diagnose and fix failures that the AI systems had introduced or couldn’t resolve. The cost of reassembling that diagnostic expertise after it had dispersed is hard to measure and harder to explain to a board that was told AI would reduce headcount costs.
Most startups aren’t at Ford’s scale. But the dynamic is the same. Smaller teams, higher stakes, less redundancy. When your engineering team consists of six people and three of them have been primarily AI-delegating for a year, the organizational resilience picture looks different than the output metrics suggest.
Mid-Career Engineers Get Hit Hardest
Junior engineers are at risk too, but there’s a specific reason that engineers five to fifteen years into their careers face a compounded problem.
They’re capable enough to use AI tools effectively. They have enough domain context to point the AI in productive directions, enough familiarity with the codebase to catch obvious nonsense. This means they delegate more work, and higher-quality work, to AI than juniors do. They extract more value from the tools.
But they’re also in the critical window when the deeper metacognitive skills, the ability to evaluate architectural tradeoffs from first principles, to look at a complex failure and reason about systemic causes rather than surface symptoms, are still being built. If that practice window fills up with AI-assisted shortcuts instead, those skills don’t fully develop.
This creates a failure mode that’s hard to catch in interviews and code reviews: the mid-career engineer who looks sharp because they can evaluate code well enough to recognize correct-looking patterns, but has lost some of the deeper capacity to reason about why it’s correct. The architectural mistake, the subtle security issue, the integration failure that doesn’t announce itself obviously, these are the things that slip through.
The engineers most at risk are exactly the ones your organization has been most enthusiastic about equipping with AI tools. The utilization pressure and the skill exposure point in the same direction.
What Actually Holds Up
The aviation industry figured out a version of this problem in the 1990s. Long-haul autopilot became standard. Pilots flew fewer manual hours. Investigations began surfacing “generalized loss of common sense and general flying knowledge” in crews who had been autopilot-dependent for years. Airlines responded not by removing autopilot, but by mandating minimum manual flying hours, by requiring pilots to demonstrate they could still do the work by hand before the automation was available to them.
The parallel for software engineering isn’t “don’t use AI tools.” That argument was lost, rightly, two years ago. The tools do accelerate real work. Abandoning them to preserve skill purity is a competitive non-starter.
The question is how you structure the work so that expert judgment stays sharp alongside the AI efficiency. Some things that actually hold up in practice:
Deliberate no-AI debugging rotations. Assign at least one production incident per month where the on-call engineer works through diagnosis without AI assistance. Not as punishment, as practice. The goal is to keep the error-encounter-diagnose-resolve cycle alive on a cadence that matches its importance.
Architecture-first prompting. Before asking AI to design a system or a data model, require the engineer to sketch the key constraints and tradeoffs by hand. The AI can then elaborate on a foundation the engineer built and understands. The comprehension gap in the Anthropic study was smallest when engineers used AI to expand on their own reasoning, not replace it.
Rotating the review. AI-generated pull requests take significantly longer to review and carry more quality risk than human-written code. Teams that rotate who is doing deep review, rather than always assigning the senior engineer, spread both the workload and the critical evaluation practice.
If you’re working with an external engineering partner, this is a reason to ask pointed questions. A team that can show you their AI-powered development practice alongside their incident response history is showing you something different from a team that shows you output velocity alone. Velocity is easy to manufacture. Diagnostic depth is not.
The 3am Test
There’s a simple frame for evaluating the health of an engineering team’s relationship with AI tools: the 3am test.
When something breaks overnight, the way it always breaks when it’s least convenient, and the person on call is working through it alone, what does that process look like? Are they forming hypotheses and testing them? Do they have a mental model of what could produce this failure pattern? Or are they sequencing AI suggestions until something sticks and then going back to sleep?
The 3am test doesn’t care about PR velocity or AI utilization rates. It cares about whether your engineers can reason.
The teams that come out of this moment in AI adoption strongest won’t be the ones who used AI most aggressively. They’ll be the ones who kept practicing the hard parts by hand, who built AI efficiency on top of genuine expert judgment rather than instead of it. That combination, fast and actually capable, is what’s worth building toward.
If you’re thinking about how to structure your engineering team for this kind of resilience, or how to bring in experienced engineers who can maintain that baseline while working at modern AI speeds, our on-demand engineering resources are built around exactly that problem. Fast isn’t enough. Fast and right is the standard that survives the 3am call.
About This Article
Two rigorous studies, one from Anthropic and one from METR, found that AI coding tools make experienced engineers measurably worse at debugging while making them feel significantly faster. This post examines why the skill being eroded is the one that matters most in production, why the engineers most exposed are the mid-career ones your organization is most aggressively equipping with AI tools, and what practices actually preserve expert judgment alongside AI efficiency.