Medicine becomes less credible when the conclusion arrives before the measurement. Healthcare AI should be particularly sensitive to that problem. Instead, the most important question in a product announcement can be the one left unanswered: what does the number actually measure?
An audience becomes adoption. Adoption becomes trust. A benchmark becomes clinical competence. A collaboration becomes validation. By the end of the retelling, several individually defensible statements can have assembled into a conclusion that nobody has properly tested.
That is the danger of marketing-based AI: the evidence may be real, but the route from evidence to promise is doing most of the work. Clinicians should be willing to call that out without pretending that every ambitious company is dishonest or every early result is worthless. The point is to make the claim earn its final sentence.
A true number can carry a false implication
Imagine a fictional clinical assistant that has processed one million questions. The count might be perfectly accurate. It could represent a broad audience, a small group of frequent users, several turns within the same unresolved task, or a mixture of clinical and educational activity.
Now place that number beside a statement about better patient care. The page has not explicitly claimed that every question improved an outcome. It has nevertheless encouraged a reader to connect the two. That connection requires evidence beyond the activity total.
The same problem applies to a test accuracy percentage. A score on a defined set may establish useful performance there. Whether the result generalises depends on the task, cases and comparison. A percentage without a clearly named endpoint is decoration with a decimal place.
The remedy is not to make every headline unreadable. It is to preserve the identity of the result: what was counted, what was tested and what decision the evidence can support.
Medicine already knows how to separate these questions
The DECIDE-AI reporting guideline addresses early clinical evaluations of AI decision support, including performance in use and human factors. Its existence reflects a basic point: testing the model and examining its interaction with clinical work are distinct parts of evaluation.
Likewise, NICE's evidence standards framework considers performance relevant to a digital technology's purpose alongside its economic impact. Neither is adequately described by the number of organisations appearing on a sales slide.
These are not demands for an impossible standard before any useful software can exist. They are ways of selecting evidence that matches the proposed use. A tool making a modest administrative claim can demonstrate that claim with a focused evaluation. A product promising to transform diagnosis has chosen a much larger burden for itself.
What deserves criticism is the flexibility of the promise: grand when attracting attention, narrow when asked for evidence, then grand again in the next announcement.
The commercial story and the clinical story need separate books
Investors can reasonably care about distribution, pricing, retention and the cost of serving customers. Clinicians can reasonably care about reliable answers, appropriate action and the work required to check the output. These interests can align, but they are not interchangeable.
A free product with rapid uptake may have found a powerful distribution strategy. That can be commercially important before a patient-outcome trial is complete. The honest description is that it has found demand under the conditions in which access is offered.
Conversely, a small product can perform a valuable task well without demonstrating a scalable business. Excellent clinical evidence does not automatically solve customer acquisition, implementation costs or renewal. A company should not be asked to prove its commercial strength with a clinical endpoint any more than it should be allowed to prove clinical benefit with a valuation.
Keeping separate books improves the conversation. It lets people be impressed by a genuine commercial achievement without recruiting that achievement as evidence for another claim.
The easiest number to collect is not necessarily the right one
Product systems naturally record actions: accounts created, messages sent, documents generated and buttons pressed. Outcomes often require more deliberate work. Did the document need correction? Did the user resolve the question? Did the recommendation improve the decision? Did anybody inherit a new task?
If an organisation rewards a team primarily for increasing recorded activity, the team has a reason to optimise activity. This is an incentive problem even when nobody intends to mislead. A feature that creates five additional interactions is easier to celebrate than one that quietly eliminates an unnecessary step.
Clinical AI needs enough measurement discipline to recognise both possibilities. More use can indicate growing value. It can also indicate a growing volume of unresolved work. Less use can indicate abandonment or a more efficient route to completion.
The important outcome should determine the event worth counting. It should not be selected retrospectively because that event happens to produce the largest graph. The existing guide to clinical AI query counts examines the difference between recorded interactions and the clinical consequences attached to them.
A favourable average can conceal the bill somebody else pays
Consider a hypothetical documentation tool that reduces a clinician's writing time by six minutes but adds four minutes of checking and three minutes of administrative reconciliation. The first measurement is a genuine reduction. The complete workflow has become one minute longer.
The relevant comparison is the work required to reach the same finished endpoint. That includes the inconvenient stages, the people outside the original study team and the cases that failed to complete. A polished generated document is an intermediate output until somebody can safely use it.
A similar issue arises when a device produces an understandable-looking health score but sends interpretation into an ordinary GP appointment. The product has delivered its result. The service has acquired a task. Whether that exchange is worthwhile depends on clinical usefulness, not the attractiveness of the dashboard.
This is why clinician involvement should shape evaluation early. The people who receive the consequences can identify work that is invisible from inside a software event log.
Independent scrutiny is more than a respectable address
An academic affiliation, a publisher relationship and an external reviewer can each improve a project. They do not answer every question about who selected the cases, controlled the data or decided what could be published.
The relevant inquiry is functional: which parts of the claim could the external party challenge, and what evidence did they see? A reviewer might independently assess outputs from a supplier-selected sample. That can be useful, but it is not the same study as independently sampling ordinary use and following the consequences.
The NIST AI Risk Management Framework's discussion of validity connects accuracy reporting to realistic test sets, methods and expected use. That is a more useful starting point than treating a familiar institutional name as a universal quality certificate.
Good scrutiny makes the role of each participant visible. It also allows favourable findings to be credited precisely, rather than either inflated into universal validation or dismissed because a company helped fund the work.
Numbers need memory when products keep changing
A product can change more quickly than its most impressive evidence. A new model, source collection, user interface or automation function may alter what the original result tells us. Carrying the old headline forward without explanation can make a historical finding appear to describe the current system.
The same problem affects growth. A company that expands from doctors to several professions, or from completed tasks to all interactions, needs a visible bridge between the old and new series. Otherwise a change in measurement can look like a change in behaviour.
An evidence history would make these transitions ordinary. Each important number would have a date, a definition and an account of material revisions. A narrower revised estimate could then be recognised as improved measurement instead of treated as an embarrassing admission.
This also changes incentives for critics. A useful correction should produce a better shared understanding, not an endless punishment for the company that explains its arithmetic.
Clinicians should refuse the role of decorative validators
There is a difference between helping build a product and lending clinical credibility to claims that have outrun the evidence. Doctors advising companies should care about both. A sound interface can be undermined by a headline that teaches users to trust it for the wrong reason.
The practical contribution is to connect each major promise to a result, a method or a clearly described development goal. If the product claims less administrative work, examine the complete work. If it claims better learning, test what users can do later without assistance. If it claims improved decisions, measure decisions rather than satisfaction alone.
iatroX's own methodology and Tutor offering should face the same distinction between a described design and a demonstrated outcome. Being clinician-built does not make the evidential questions disappear.
Healthcare AI can be ambitious and exact at the same time. A company can say that a result is promising, explain precisely why, and state the next claim its evidence does not yet establish. That is a stronger account of progress than asking one impressive number to stand in for an entire care pathway. Medicine should insist that the measurement survives the marketing.
Frequently asked questions
Does criticism of AI marketing mean opposing healthcare AI?
No. Accurate claims help clinicians identify useful products and help good suppliers compete on what they actually deliver. The criticism concerns the distance between a measured result and the promise attached to it.
Should every healthcare AI product have a randomised patient-outcome trial?
The evaluation should match the task and the claim. A bounded administrative improvement and a claim about treatment decisions require different evidence. What matters is whether the chosen design can answer the question being advertised.
What is the fastest way to assess an impressive AI number?
Identify the measured endpoint and finish this sentence: this result establishes what, for whom, under which conditions? If the company's headline requires an additional conclusion, look for the evidence supporting that step.
Explore clinical questions and their linked sources with Ask-iatroX ->