AMIE's reported 90% diagnostic accuracy means that a sufficiently matching diagnosis appeared within its top seven suggestions. It does not mean the first diagnosis was correct in 90% of encounters. The March 2026 preprint reports approximately 56% first-choice accuracy and 75% top-three accuracy, figures also contextualised in Google's account updated on 8 October 2026.
Those measures answer different questions. A clinical team evaluating a differential generator may value a broad list that includes an important possibility. A patient deciding what to do next needs prioritisation, explanation and an appropriate plan. The same percentage cannot describe all those functions.
One case can count as correct under one measure and wrong under another
Imagine a fictional teaching case in which a patient's eventual diagnosis appears sixth in an AI-generated list. The system has not placed it first or within its first three suggestions, but it has included it within seven. Under top-one scoring, that case is unsuccessful. Under top-three scoring it remains unsuccessful. Under top-seven scoring it succeeds.
Nothing about the patient changed between those calculations. Only the question asked of the output changed. The example is a scoring illustration, not an observed AMIE consultation.
The detailed diagnostic figure in the March 2026 preprint reports the following for the patients who completed the AI interaction and subsequent appointment:
| Measure in the March 2026 analysis | Reported count | Approximate headline | Interpretation |
|---|---|---|---|
| Top one | 55 of 98 | 56% | The first suggestion met the study's diagnostic matching threshold |
| Top three | 73 of 98 | 75% | A matching suggestion appeared within the first three |
| Top seven | 88 of 98 | 90% | A matching suggestion appeared within the first seven |
These are alternative summaries of the same analysed cohort, not three independent studies. The percentages are approximate presentations of the reported counts.
What counted as a diagnostic match
Another detail matters beyond the length of the list. The March 2026 methods used the Bond/Graber scale. A suggestion could qualify if it was the actual diagnosis or something judged very close, rather than requiring an exact diagnostic label in every successful case. Two raters assessed the diagnostic lists and resolved disagreements. The preprint methods describe this process.
That approach can be defensible. Clinical language is not always standardised, and penalising every near-equivalent label could misrepresent useful reasoning. However, it means that the most precise description is performance against the study's matching rule, not a claim that every successful output contained an exact, independently proven disease name.
When comparing another system with AMIE, ask whether it was allowed the same list length and the same matching threshold. A first-choice exact-match score cannot be placed directly beside a top-seven score accepting near matches. A table that ignores those differences would create the appearance of a league table without a common competition.
Where the eventual diagnosis came from
The reference diagnosis was established through chart review eight weeks after the appointment. In the March 2026 methods, a panel of internists reviewed longitudinal records while blinded to AMIE's output, including later investigations and specialist information where available. The diagnostic methods therefore used more than the initial appointment label.
Nevertheless, a chart-derived final diagnosis is not necessarily a test-confirmed diagnosis. The researchers separately examined a subset of 46 patients whose diagnosis was supported by a diagnostic test. Google's October 2026 account makes that distinction explicit. The overall top-seven result should not be described as if every diagnosis in the analysed cohort had laboratory, imaging or pathological confirmation.
Nor does an eight-week reference point necessarily settle every conceivable uncertainty. It identifies the study's follow-up window. A future study could reasonably use different reference standards for different presentations, but it would need to explain those choices before readers compared the scores.
Including a diagnosis does not complete the clinical work
A useful differential changes what the clinician considers. A useful management decision determines what should happen for this patient. The transition is not automatic.
| A system can include the correct diagnosis and still need help with... | The unresolved clinical question |
|---|---|
| Ranking | Should this possibility lead the assessment or remain a remote alternative? |
| Discrimination | Which unanswered question or examination finding would change its plausibility? |
| Proportionality | Is an investigation justified now, later or not at all? |
| Communication | How should the possibility be explained without overstating certainty? |
| Follow-through | Who reviews the result or reassesses a changing presentation? |
This is an editorial framework for reading diagnostic evidence, not a list of failures observed in AMIE.
Return to the fictional sixth-ranked diagnosis. If the first five suggestions lead to unnecessary activity, the correct sixth entry does not erase that burden. Equally, if its presence prompts the clinician to ask a decisive missing question, the lower ranking may still have practical value. Both consequences would require observation rather than inference from a top-seven score.
A more useful diagnostic-AI scorecard
A prospective comparison should retain ranking measures while adding consequences. Report the first-choice result, the performance of a bounded differential and the rate of important omissions separately. Then examine whether incorrect suggestions prompted unnecessary action, whether serious possibilities were communicated appropriately and whether clinicians changed their decisions for sound reasons.
The scorecard should also preserve the eligible population. Excluding patients who could not complete intake might be appropriate for a particular model-performance calculation, but a service-level evaluation should still explain what happened to them. A technically strong result among completers and an accessible service are related, different achievements.
This distinction follows the broader separation between model outputs and live clinical use emphasised by DECIDE-AI's early-evaluation reporting framework. The framework is not a certification or a replacement for a suitable clinical study.
How to use the number without dismissing it
For a researcher, the top-seven result is evidence that a supervised conversational system could elicit enough information to generate a relevant differential in many analysed encounters. For a clinician, it is a reason to inspect how the list was formed and how it might help, not permission to skip assessment. For a patient, it is not a personal probability that the first explanation offered is correct.
A simple learning exercise is to take a fictional differential and identify the additional information needed to change its order. iatroX's clinical reasoning tools can provide an educational setting for that exercise; AMIE's results do not validate iatroX or any other platform by association.
Frequently asked questions
Was AMIE's first diagnosis correct in 90% of cases?
No. The March 2026 analysis reported approximately 56% first-choice performance, while the approximately 90% result concerned a matching diagnosis within the top seven suggestions.
Were all the eventual diagnoses confirmed by a diagnostic test?
No. The researchers distinguished the full chart-reviewed cohort from a test-confirmed subset of 46 patients in their March analysis and October 2026 account.
Does top-seven accuracy show that an AI can manage the patient safely?
No. It describes diagnostic-list performance under a particular scoring rule, not the appropriateness, communication or consequences of a complete treatment plan.
Practise turning a differential into discriminating questions →