The most dangerous word in a healthcare AI pitch may be the one after the number.
A million what? Ninety-four per cent accurate at what? Validated for which decision? Used by which clinicians, during which period, and doing what? These are short questions. A credible product team should have short, intelligible answers before the discussion becomes technical.
Clinicians already know how to question evidence. The difficulty with AI marketing is that a sales page can move between several kinds of evidence faster than a reader notices: a large training collection becomes a performance claim, a performance score becomes clinical reliability, and clinical reliability becomes a promise of better care.
A practical appraisal interrupts that movement. It keeps each claim attached to the thing actually measured.
Start with the clinical job
Before opening a benchmark paper, finish this sentence: the product is supposed to help this user perform this task, in this setting, with this output.
That sentence prevents several distractions. A product that summarises literature needs to be assessed for the quality and support of its summaries. A documentation tool needs evaluation of the note it produces and the work required to correct it. An educational tool needs a learning question, not a borrowed claim about diagnostic performance.
The same underlying model could contribute to all three products without making their evaluations interchangeable. The surrounding sources, interface, instructions and human workflow are part of what the clinician uses.
If the intended job remains vague after a demonstration, the evidence discussion has started too early. First establish what success would look like at the point of use.
Translate the six favourite claims
The following table is a reading aid, not a verdict on any particular vendor. Its purpose is to expose the missing noun or qualification before it becomes an assumption.
| Marketing phrase | Immediate follow-up |
|---|---|
| Used by | Which people, performing which action, during which period? |
| Trained on | What development material was used, and what separate data tested performance? |
| Accurate | Which output was compared with which reference, using which error measure? |
| Validated | For what population, setting, product version and intended use? |
| Partnered with | What does the partner actually provide or independently assess? |
| Improves care | Which measured patient or clinical outcome changed, compared with what? |
The answer should narrow the claim. If it merely replaces one broad adjective with another, there is more reading to do.
For example, saying a system is clinically validated does not identify whether investigators tested answer quality, diagnostic classification, user satisfaction or patient outcomes. Each can be a legitimate endpoint. Each answers a different question.
Read the denominator before admiring the percentage
An impressive percentage can describe a common class, a selected subgroup or only the occasions when a system chose to produce an answer. The reader needs to know which observations entered the calculation and which disappeared before it was made.
Consider a deliberately hypothetical service claiming 95% accuracy. Suppose it produces an answer in 20 of every 100 eligible cases and is correct in 19 of those 20. Its accuracy among answered cases is 95%. Its coverage is 20%. Both numbers are necessary to understand the service.
The unanswered cases might reflect sensible restraint, a technical limitation or missing information. Their existence does not establish poor design. Hiding them would prevent a buyer from understanding whether the product can do the proposed job.
The same habit applies to subgroup results. Ask whether the headline includes the users most relevant to your setting, and whether enough of them were evaluated to support a useful conclusion. An overall average cannot supply information that was never collected.
Benchmarks deserve specific criticism
A benchmark can be substantial without answering every clinical question. OpenAI's HealthBench uses 5,000 simulated health conversations and physician-written assessment criteria developed with 262 physicians. Its assessment covers accuracy as well as behaviours such as seeking context and handling uncertainty, with a model-based grader evaluating responses against the criteria.
Reducing that evaluation to conversational politeness would miss what it measures. Treating a high score as proof of improved outcomes in an NHS clinic would go beyond what the task establishes. The useful appraisal identifies the tested capabilities and then examines what evidence is needed for the proposed application.
Ask whether a reported score is overall or restricted to a theme. Check whether products were tested under comparable conditions. Look for uncertainty around differences and whether the important errors are visible. A tiny advantage in a summary score may matter less than a recurring failure on the task you need performed.
The CONSORT-AI reporting extension reinforces why intervention details, human interaction and error analysis belong in reports of clinical AI trials. The brand name and final score cannot carry the whole interpretation.
Follow one recommendation back to its source
For evidence-answering products, a useful practical check is to select a consequential recommendation and open the material cited for it. Identify the passage that supports the recommendation. Then ask whether the source addresses the same population and question.
This is more revealing than simply counting references. A real reference can be irrelevant. A relevant paper can be overstated. A source can support an association while the answer presents a treatment instruction. A page full of citations does not establish which claim each one supports.
The check also exposes the cost of verification. If the answer takes seconds to generate but requires a lengthy search to establish what its references actually say, that work belongs in the assessment of usefulness.
When examining iatroX, readers can use the product explanation and methodology as starting points. The relevant test remains the answer in front of them and whether its support can be inspected.
Run a five-minute appraisal before committing to a longer one
Here is an illustrative sequence for an initial review. In the first minute, write down the exact claim and intended task. In the second, identify its original source and date. In the third, establish the population, denominator and outcome. In the fourth, examine a representative output or failure example. In the fifth, decide which unresolved question would change your willingness to use or evaluate the product.
This is a screening exercise, not a substitute for a full procurement assessment. Its value is to prevent hours being spent discussing the wrong evidence.
Suppose a demonstration of a clinical assistant produces an elegant answer but offers only a general benchmark score when asked about source support. The next useful action is a task-specific assessment of supported recommendations. Asking for another testimonial or a larger user number would leave the original question unresolved.
Conversely, a team that provides a clear endpoint, inspectable methods and representative failures has made a more productive conversation possible. Its claim may be narrower than a rival's headline, but it gives the evaluator something concrete to work with.
Decide what the evidence permits
The outcome of appraisal need not be a binary declaration that the product works or does not work. Evidence might justify a limited evaluation, a particular educational use or a defined clinical workflow while leaving other applications unsupported.
Write that scope plainly. State the task the evidence supports, the conditions under which it was assessed and the remaining question that matters. That is more useful than an all-purpose endorsement.
The clinician's BS detector does not require cynicism. It requires refusing to let a large number complete a sentence that the underlying evidence has left unfinished.
Frequently asked questions
What is the fastest way to assess an AI accuracy claim?
Identify the output, reference standard, denominator and excluded cases. Then check whether the evaluated population and task resemble your intended use. An accuracy percentage without those details is an incomplete description of performance.
Does a disclaimer mean the product has no useful safety behaviour?
No. Wording about intended use does not establish how the system behaves. Assess observable responses to missing context, uncertainty, unsupported questions and relevant errors. The product's behaviour and the evidence about it need to be examined directly.
Should every medical AI answer contain a source link?
The requirement depends on the task. For a product presenting evidence-based clinical recommendations, traceable support is a central evaluation question. A drafting or administrative task may need different checks. In every case, assess whether the output can be verified in the way its intended use requires.