A study of four Polish endoscopy centres found that doctors who spent three months with an AI polyp detector got worse at finding polyps without it. Adenoma detection in standard colonoscopies went from 28.4% before the AI arrived to 22.4% after. Nobody in that building was replaced. The AI worked. The humans just stopped doing the part the AI was doing, and the part quietly atrophied.
This week a post from SigNoz applied that same pattern to on-call engineers, and it reached the Hacker News front page. The argument is that AI is not replacing SREs, it is deskilling them. The comments split exactly where you would expect: half agreed, half said it is just the calculator and long division again.
Both camps are arguing about the wrong thing. The interesting question is not whether deskilling happens. It is why your team will not see it happen, and the answer has been sitting in a 1983 paper for forty years. The thesis of this issue is simple: you only lose skills you have stopped measuring, and every AI rollout I have seen measures the assisted number and never the unassisted one.
What the evidence actually says
Start with the colonoscopy study, because it is the cleanest. It was published in The Lancet Gastroenterology & Hepatology in 2025. It was observational and retrospective: 1,443 patients, 795 before AI and 648 after, across four centres. The primary outcome was the detection rate of colonoscopies done without the AI switched on. That detail matters. The researchers did not measure whether the AI helped. They measured what happened to the human when the AI was off. The adjusted odds ratio was 0.69, the absolute drop was 6.0 points, and the confidence interval ran from 1.6 to 10.5 points. The authors are careful to say this shows an association, not proof of cause.
Now the coding trial. Anthropic's alignment team ran a randomized controlled trial with 52 mostly junior engineers learning Trio, a Python async library. One group coded with an AI assistant for 35 minutes, the other by hand, then both took a quiz without help. The AI group averaged 50%, the hand-coding group 67%. The AI group was about two minutes faster, but that difference was not statistically significant. Read that again: they gave up a lot of understanding and got almost no speed in return. The study also found that people who used the AI to ask conceptual questions, or who read the code it generated and asked for explanations, kept most of their learning. The ones who delegated and moved on did not. Sample sizes for those sub-patterns were tiny (2, 3 and 7 people), and the authors say the analysis is qualitative, so treat it as a lead, not a result.
Then aviation. A 2019 flight safety study put 20 Airbus A340 pilots, averaging about 11,600 flight hours, through night approaches in a simulator with the flight director on and off. With it on, flight paths were more precise and pilots looked less at the outside world and the raw altitude instruments. The authors concluded that flying with partial automation cannot replace flying without it for keeping basic skills. Note the shape of the finding: the assisted pilots looked fine, and the evidence of change was in where they pointed their eyes.
Three fields, three measurements, one pattern. Performance with the tool on goes up or holds. Performance with the tool off, which is the performance you need on the worst day, slides.
The irony is that you will not notice
Lisanne Bainbridge saw this coming. Her paper Ironies of Automation, published in the journal Automatica in 1983, is among the most cited in human factors research. Its core argument is that automating most of a job leaves the human with the residual tasks the machine cannot handle, which are the hardest ones, while removing the daily practice that built the skill to handle them. As one summary puts it, she also noted that automatic control can compensate for a developing fault so well that the problem is invisible until it is beyond control, and that a system should fail obviously rather than degrade gracefully.
Apply that to an AI-assisted incident workflow. The assistant triages the alert, correlates the signals, proposes the rollback. Mean time to resolution drops. The dashboard is green. And the on-call engineer's ability to form a hypothesis from raw logs at 3 a.m. with the assistant's vendor having its own outage is decaying, with no instrument pointed at it.
This is the same failure mode as a retry loop that hides a flaky dependency. The graceful handling is the problem. Your MTTR chart is the equivalent of the flight director being on: everything about the assisted path looks great, and the evidence of erosion lives in the one place you are not looking.
The metric nobody tracks
So here is the proposal, and it is deliberately boring. Track an unassisted baseline, the way the colonoscopy researchers did. Pick a small set of skills you cannot afford to lose, and measure them with the tooling off, on a schedule.
For an on-call rotation, that means two numbers. First, time to first credible hypothesis from raw telemetry with no assistant. Second, the fraction of engineers who can reach a correct root cause on a standard drill without help. You do not need a research design. You need the same drill, run quarterly, scored the same way.
The cheapest way to generate practice is to make manual handling a policy, not a mood. Here is a small sketch that deterministically picks which incidents get handled by hand, so it cannot be gamed by whoever is tired that week:
import hashlib
MANUAL_RATE = 0.10 # one in ten eligible incidents is handled with no assistant
def handle_manually(incident_id: str, severity: int) -> bool:
if severity <= 2: # never practice on the real fires
return False
h = int(hashlib.sha256(incident_id.encode()).hexdigest(), 16)
return (h % 1000) / 1000 < MANUAL_RATE
Severity one and two incidents stay fully assisted, because you are not going to slow down a real outage for a training goal. The sampled low-severity ones get a no-assistant rule, and the engineer writes down their first hypothesis before looking at anything. Log that timestamp. After two quarters you have a trend line for the thing that was previously invisible.
This is more targeted than the usual advice to be deliberately inefficient. Inefficiency across the board is a cost with no feedback. A sampled drill with a number attached tells you whether the skill is holding.
The yes, but
The strongest objection came up in the Hacker News thread, and it is a fair one: this is just trans-skilling. People stopped doing long division when calculators arrived and nothing bad happened. The skill moved up a level. Attention is a choice.
Part of that is right. Nobody needs to hand-write boilerplate to be a good engineer, and I am not arguing that every abstraction destroys the craft. If the lost skill is irrelevant on your worst day, let it go.
But the calculator analogy breaks in two places the commenters already named. A calculator is deterministic, so you do not need to verify its output with the skill it replaced. A model is not, so the person checking it needs the underlying knowledge, and the Anthropic trial suggests that knowledge is exactly what drops when you delegate. The second break is the failure cost. A dead calculator is an inconvenience. A dead assistant during an outage, when you are on call for a system you only ever debugged through that assistant, is the incident.
There are also honest limits on the evidence. The colonoscopy study is observational and ran over three months. The coding trial had 52 people and measured one session, not a career. The aviation work used 20 pilots and reports significance without effect sizes in the slides I could read. None of this proves that your SRE team is getting worse. It proves the mechanism is real in three unrelated domains, which is enough to justify a cheap measurement.
What to do on Monday
Pick one skill your team could not afford to lose, and run it with the assistant off. Not as a punishment, as a baseline. Score it, write the number down, and put a date on the next drill. If the number holds, you have paid an hour to learn the rollout is safe. If it falls, you found out in a drill rather than in an outage.
The deeper habit is to ask, for every AI tool you adopt, what the human no longer does and whether anyone is still measuring it. The colonoscopists were skilled, motivated and well supervised, and they still slipped by six points without a single person deciding to get worse. A skill you do not measure will not announce when it leaves.
