Six months ago, every answer I got from the AI came with its reasoning attached: what it checked, what it compared against, what it decided to flag. Lately the answers are two words: “Looks right.” “Shipped it.”
Nobody told the AI to stop explaining itself. It just got confident enough to stop asking.
Call it AI watching AI, because that is where this always ends up. The AI got faster at sounding sure, not at being right, and the fix nobody says out loud is more AI: a second one checking the first, and increasingly a third checking that.
That is not the story anyone was sold. Superintelligence was supposed to need less checking, not more layers of AI doing the checking for it.
If getting smarter actually meant getting more careful, the checking would shrink over time. Instead, it’s stacking.
Why does smarter AI need more AI to watch it, not less?
Early on, the AI had no track record, so every answer came with a visible check attached: what it compared, what it flagged, what it confirmed. As it got better, the track record did the talking instead of the checking.
Confidence crept in exactly where the verification used to be. A little at a time, the AI that used to say “confirmed against the spec” started saying “shipped it,” and nothing about its actual accuracy improved to match the new tone.
That’s the trap, and it isn’t only the AI’s. Automation bias is the well-documented habit of trusting polished output more, not less, which means the people reading those answers stopped double-checking at the exact same moment the AI stopped showing its work.
A more confident AI reads as a more capable one, and the two are not the same claim. So the fix wasn’t a more careful AI, it was a second one, whose entire job is catching what the first one skipped.
If AI isn’t getting more careful, why do we keep adding more of it?
This is where the superintelligence pitch quietly changes shape. The promise was one AI, smart enough to need nobody watching it. What’s actually showing up is a stack: AI generating the work, a second AI reviewing that work, sometimes a third AI judging whether the second one’s review was any good.
Using one AI to grade another AI’s output is now a standard evaluation method, not a stopgap. It does not mean the first AI got smarter, only that checking it got easier to automate.
So what actually catches what AI skips?
Not another AI with slightly more confidence than the last one. A step that runs on a fixed rule, whether or not anything involved feels sure of itself.
Code review exists for exactly this reason: not because the person doing the work is bad at it, but because a second, structural look catches what a confident first pass won’t. It doesn’t need to distrust the work, it just needs to run every time, regardless of how sure anyone or anything sounds when they hand it over.
The WHO Surgical Safety Checklist proves the same point outside software: a fixed two-minute check cuts complications and deaths by over 30 percent, not because surgeons got less skilled but because a rule that runs every time catches what confidence won’t.
Stacking more AI on the problem feels like progress. Each new layer looks like intelligence catching its own mistakes. The part that doesn’t show up in the pitch is that none of those layers are actually getting smarter, they’re just getting cheaper to add, and a fixed check still catches what a hundred confident layers won’t.
Previously: The drinking bird got promoted