Checking the checkers: what a human check of an AI-built evidence map found, and missed
A human check is the usual safeguard for AI output, but a check is an instrument too. Two medical students checked 85 items of an evidence map built with an earlier version of Julie against the source papers. Of their 51 flags, the adjudicator upheld 8, and the flags gathered where the machine’s own explanation disagreed with its answer rather than where the errors were.
The map was built in early 2026: thousands of quotations from the literature, each linked to a recommendation and labelled with a study type, a population, a level of evidence and a grade. In April, two medical students, given a guide to the Oxford CEBM levels, checked 85 of its items against the source papers, seven things for each: whether the quotation was in the paper, the study type, the population, the level, the grade, the relevance, and how the passage bore on the recommendation. That made 595 cells. The items came from an earlier AI audit of the map, weighted towards those the AI auditors had questioned. Beside each item the students saw the values under review and the explanation the machine had given for them.
| What we counted | Cells |
|---|---|
| Checked against the source papers | 595 |
| Flagged as wrong by the students | 51 |
| Flags upheld by the adjudicator | 8 |
| Flags overturned by the adjudicator | 43 |
| Changed in an earlier correction round, passed by the students | 16 of 23 |
The students found every one of the 85 quotations in the paper it was attributed to. Their flags tell a different story. The 43 overturned flags fell on 21 items from three papers, 36 of them on one paper. On the level of evidence, the students flagged 20 of the 21 items whose explanation named a level other than the one shown, and none of those flags was upheld. On one paper the value shown was 4, its explanation discussed higher levels, and the students proposed 3. On the other, their correction gave what the machine’s explanation said: a retrospective cohort, level 2b, grade B. Of the 64 items whose explanation named no other level, they flagged two, and both flags were upheld. Meanwhile 23 cells in the sample had been changed in an earlier correction round; the students flagged 7 and passed 16, all of them study types, populations or a relationship rather than levels.
What this does and does not show. This is one check, and its reference is weaker than we would want. A single adjudicator reviewed the flags: a senior clinician who also worked on the grading and works on Julie. The cells the students passed were not reviewed again. The paper behind most of the overturned flags is a hard case under the Oxford levels. The two students returned one shared set of answers, so their agreement with each other cannot be measured. The changed cells are counted against an earlier correction round, made after an AI audit and checked against the papers. The sample is small, clustered and weighted towards questioned items, and the pattern rests on two papers: it is an observation from one check, not an error rate for the map, and not a rule about human checkers.
What we take from it. A check measures the machine only as well as the check is built. Here the flags gathered where the machine’s explanation signalled doubt, while most of the cells changed earlier passed without one. A person checking AI output while looking at the AI’s answer and its reasoning is not an independent reference, however careful. The checks we run on Julie now are built the other way round. In the validation of her screening, two reviewers label every record blind to each other’s work and to all machine output, and the human check Julie prepares for a review can use a blind sheet, which shows a reviewer neither her decision nor anyone else’s answer.
Tell us about your project, and what you would expect from Julie: what she should do for you, and how reliable she would need to be.
All news