A memorable statistic can frame an entire product announcement. The suggestion that roughly one in four medical questions cannot be answered by research makes the case for expert clinical judgement feel immediate. It also invites a necessary question: one in four of which questions?
The distinction does not diminish the importance of evidence gaps. It makes the argument more useful. Before a research finding becomes a claim about every consultation, its unit of analysis and sampling frame need to remain intact.
What the cited research counted
The medRxiv meta-research study examined 64,849 population-intervention-comparison-outcome questions from 5,116 Cochrane intervention reviews. It reported no eligible studies for 24% of those questions. The source is a preprint, and the finding concerns that defined collection rather than a representative sample of everything clinicians ask.
The earlier full-text version describes the use of reviews containing summary-of-findings tables and distinguishes questions without studies reporting the outcome. This detail helps explain why the denominator is not interchangeable with all medical uncertainty.
A question about an intervention's effect on a particular outcome is different from a question about anatomy, a diagnostic definition, a local referral route or the contents of a product label. They may all arise in clinical work, but they do not belong to the same measurement exercise.
Why the denominator changes the conclusion
Suppose an original educational review asks whether an intervention changes hospitalisation, quality of life and mortality. These are distinct outcome questions even when they concern the same intervention and population. Evidence may be available for one and absent for another.
Counting unanswered outcome questions therefore tells us something important about the completeness of evidence. It does not tell us how often a doctor opens a reference and finds no useful information. The clinical question may be broader, narrower or organised differently.
Similarly, the proportion of unanswered questions cannot be applied directly to consultation volume. A question that appears once in a review and a question asked thousands of times in everyday practice would need different weights in a study of clinicians' actual information needs.
No eligible study is not the same as no useful knowledge
Eligibility criteria define what evidence can answer the study's specific question. Material outside those criteria might still inform background understanding, mechanism or a related decision. That does not mean excluded evidence should be treated as equivalent to a suitable trial. It means absence within a defined analysis should not be translated into absence of every relevant consideration.
The reverse mistake is equally important. The existence of a study does not establish a dependable answer. A small or poorly applicable study can leave substantial uncertainty. A binary distinction between research present and research absent would hide that problem.
For clinical AI, the lesson is to describe the type of uncertainty. Is the evidence missing, indirect, conflicting, imprecise or simply not retrieved? Those situations call for different explanations and different next steps.
Questions within a review are not necessarily independent
Several outcome questions can arise from the same review or related evidence base. A percentage calculated over those questions should therefore not be interpreted as thousands of wholly independent demonstrations of the same problem.
This matters when a statistic is used rhetorically. A large denominator can sound like a direct census of clinical practice, even when the underlying design is a structured analysis of evidence synthesis. The study remains valuable on its own terms. The error would be making it answer a different question without additional data.
A more precise public interpretation is that substantial gaps remain in the evidence available for specified intervention-outcome questions. That is already a serious finding; it does not need to be expanded into a universal claim to matter.
Expert synthesis helps navigate uncertainty, not abolish it
When direct evidence is incomplete, clinicians still have to reason. An expert-authored account can help distinguish accepted practice, indirect evidence and unresolved questions. Its value is in making the basis of judgement intelligible.
The danger would be presenting expert synthesis as though it has converted an evidence gap into experimental certainty. Experience can inform a recommendation without changing the underlying strength of evidence. A responsible answer should preserve that distinction.
This is the strongest argument for maintained reference content: it can organise what is known and explain how practitioners reason where knowledge is limited. It is a weaker argument for saying that access to a particular product has solved the evidence gap itself.
What an AI answer should reveal
A useful response to an uncertain question would identify the decision, explain the relevant evidence boundary and state what is being inferred. It would avoid supplying a confident answer merely because the interface expects one.
It would also distinguish a retrieval limitation from a research limitation. "I did not find a source" is not equivalent to "no source exists". Establishing the latter can require a more systematic search than an ordinary chatbot interaction provides.
For readers comparing tools, these distinctions are observable. Does the response identify uncertainty precisely? Does it show what supports the recommendation? Does a follow-up preserve the uncertainty or gradually turn it into an unqualified instruction?
The relevance to iatroX
A focused reference question can be a useful way to make uncertainty explicit. iatroX's explanation of its methodology provides a basis for examining its source-grounded approach, but the actual answer should still be judged on whether it represents the evidence faithfully.
The educational opportunity is equally important. A clinician who learns to distinguish absent evidence, indirect evidence and absent retrieval becomes better equipped to question any AI output. That skill remains valuable regardless of which publisher or platform supplies the first answer.
The statistic's strongest use is therefore not to end the debate with a brand name. It is to encourage a more precise account of what we know, how we know it and where judgement remains necessary.
Frequently asked questions
Does the study show that one quarter of all questions doctors ask are unanswerable?
No. It analyses defined intervention-outcome questions in a particular collection of Cochrane reviews. Applying that proportion to all clinical questions would require a different study and sampling strategy.
Does a lack of eligible studies mean clinicians have no basis for action?
Not necessarily. Other forms of knowledge or indirect evidence may inform judgement, but their role and limitations should be stated rather than presented as equivalent to direct, high-certainty evidence.
Does the study establish which clinical AI product is best?
No. It is evidence-gap research, not a head-to-head product evaluation. A product's ability to represent uncertainty requires its own assessment.
Explore Ask-iatroX with a question that makes the evidence uncertainty explicit.