The new BMJ study on chronic kidney disease gives clinicians a practical reason to take guideline-grounded AI seriously. It also shows why choosing a clinical assistant requires more than asking which model powers it. The source collection, retrieval method, handling of patient context and presentation of the answer are all worth evaluating.
For iatroX and similar clinical knowledge platforms, the opportunity is to make authoritative information easier to find, inspect and use. Our view is that this becomes more valuable when a useful answer can also lead to better understanding and retained learning. This is iatroX editorial analysis; we have a commercial interest in iatroX. The study below evaluated research pipelines, rather than iatroX or a commercial product comparison.
What the published BMJ paper reports
Sesen, Yeung and Asgari's paper, published on 2 November 2026, compared GPT-4o alone, vector RAG and GraphRAG on nine questions for each of 70 synthetic CKD cases. The indexed source was a single NICE CKD guideline. This was a synthetic-case evaluation, not a prospective patient study.
The authors report overall clinician-rated correctness of 70.8% for baseline, 76.6% for RAG and 75.6% for GraphRAG. Reported unadjusted p values against baseline were .045 and .057 respectively; RAG versus GraphRAG was .897. These are the paper's reported aggregates, not independently reconstructed counts. The detailed per-question table does not transparently reconcile with those aggregates, which warrants clarification before reusing them as a benchmark.
GraphRAG received higher patient-specificity and lower clarity ratings from an LLM judge, rather than a second clinician. One nephrologist developed reference answers and rated correctness. Prompting and pipeline settings differed, limiting a clean architecture-only comparison. The findings support further investigation of retrieval, rather than establishing a universal GraphRAG advantage.
Treat the source collection as part of the clinical product
A general model draws on information learned during training. A retrieval-based system also selects material from an external collection and uses it while answering. iatroX's methodology describes retrieval, source prioritisation, citation grounding and orchestration as parts of its approach. This helps explain why the name of the underlying model is only one part of a meaningful product description.
Imagine a clinician asking two questions about the same condition. One asks how an established guideline defines a category. The other asks whether a newer intervention changes the options available. These questions can require different documents, even though both sit beneath the same disease label.
For a local evaluation, build a source map before scoring answers. Identify the guideline, related updates, relevant safety information and local pathways that would normally inform the selected task. Then ask the supplier how the system handles a question whose necessary source falls outside its collection.
This creates a constructive test for iatroX and other referenced platforms. An answer should give the clinician a route to the material that supports it, while preserving the boundaries of that material. A useful limitation or an appropriate route to another source can be more valuable than a fluent attempt to fill a gap.
Distinguish finding information from applying it
Locating a paragraph is a different achievement from deciding whether its conditions fit the question. A recommendation may depend on a population, setting, previous intervention or an exception described elsewhere. A good reference experience should make those dependencies easier to inspect.
Use a fictional test question with two otherwise similar scenarios. Change one relevant feature, keeping the rest constant. Before testing, have the reviewing clinician specify whether the expected response should change and why. Then inspect the retrieved material as well as the final answer.
The purpose is to identify where the workflow helps. If both outputs cite the same paragraph but only one is applicable, the problem lies beyond simple document access. If a decisive source never appears, the retrieval stage needs attention. If the relevant material appears but its qualification disappears in the summary, the synthesis needs attention.
This is an original evaluation proposal, not a description of a demonstrated iatroX feature or a finding about a particular supplier. It gives clinicians a way to discuss performance in terms of observable work rather than an architecture label.
Make readability a clinical usability requirement
A lengthy response can contain useful material while leaving the reader unsure which point matters. A very short answer can be easy to read while losing the condition that makes it applicable. The practical objective is a clear answer with enough evidence and qualification to review.
During a pilot, ask clinicians to identify three things without another prompt: the main conclusion, the circumstance that could change it and the supporting source. Record how long this takes and where interpretation remains difficult. Assess the complete reading and checking process, rather than response-generation time alone.
Avoid treating more patient details as proof of a more useful answer. A response can repeat information from the question without explaining why it matters. Ask the reviewer to name the detail that actually changes the interpretation. If none does, the additional wording may be decoration.
Conversely, a concise answer should retain the important uncertainty. Requesting a shorter version is useful only if the clinician can still see the scope of the conclusion. A product's ability to support both a quick orientation and deeper source inspection is worth testing directly.
Evaluate knowledge answers and calculations differently
Consider a fictional query that combines an explanation of a recommendation with a numerical calculation. These are separate tasks. The explanation requires an applicable source and a faithful interpretation. The calculation requires correct inputs, units, formula and implementation.
For such a local test, use a trusted calculation method to establish the expected numerical result. Check whether the AI identifies missing inputs, makes assumptions visible and keeps the explanatory answer separate from the arithmetic. A persuasive explanation does not independently verify a number.
This approach also prevents a misleading overall score. A tool may perform well at finding definitions and less well at operations requiring precise computation. Report results by task, including the consequence of an error and the work needed to discover it. Choose the workflow around those observations.
Where iatroX offers a strong practical proposition
iatroX brings focused Ask queries and structured Brainstorm reasoning into a platform that also supports deliberate learning. A clinician can formulate a specific question, inspect the cited material and use the resulting uncertainty as the starting point for further study. That continuity is a useful reason to consider it alongside other clinical knowledge services.
The strongest assessment starts with questions clinicians actually revisit. Select a small set, establish what a useful answer requires and judge whether the tool helps the reader reach that point. Include source applicability and unresolved uncertainty in the assessment. A quick answer is valuable when it reduces the work needed to reach a defensible understanding.
The wider market offers other approaches to referenced assistance. UpToDate Expert AI and AMBOSS AI Mode should be compared through their current documented sources and intended workflows. A familiar publisher, a source link or a retrieval architecture is a useful starting point for evaluation; the relevant question is how well the complete service supports the selected task.
None of these platforms needs to be identical to the BMJ prototype to have value. The research gives buyers better questions to ask about product design and evidence. It does not supply a transferable accuracy percentage for a platform that was not tested.
Turn a recurring clinical question into retained understanding
An answer can resolve today's search without making tomorrow's reasoning easier. If the same distinction keeps generating uncertainty, protect a short period for active practice rather than repeatedly accepting another explanation.
iatroX Tutor starts from a question the learner has already attempted. It supports guided reasoning and a direct explanation, with editable learning cards available for later review. iatroX CPD connects focused questions, discussion, personal reflection and a reviewed learning record. Continued Tutor and new CPD learning after the introductory experiences use all-access; current access information explains the arrangements.
An original learning routine is straightforward: state the uncertainty, inspect the relevant evidence, attempt a supported question, explain the distinction in your own words and revisit it later without assistance. Judge the routine by what you can subsequently explain and apply. A saved answer or completed record describes an activity; later reasoning provides another kind of evidence.
That is a positive case for iatroX which the research helps put into context. A maintained reference workflow can support the immediate question, while structured learning gives the clinician a route to become more capable at the next one.
The considerations to take into a product trial
| Consideration | A concrete question to answer locally |
|---|---|
| Source coverage | Is the necessary document included, current and appropriate to this setting? |
| Retrieval | Can the reviewer locate the material supporting the answer? |
| Application | Are the recommendation's conditions and exceptions preserved? |
| Readability | Can the clinician identify the conclusion and uncertainty efficiently? |
| Task boundaries | Are calculations and actions evaluated separately from explanation? |
| Learning | Can a recurring gap become practice and a later check of understanding? |
Use these questions to choose a service for a defined purpose. The worthwhile ambition is better access to clinical knowledge and a better supported clinician, with evidence gathered at the level where the tool will actually be used.
FAQs
Does this BMJ paper show that iatroX is more accurate than a general chatbot?
The study did not evaluate iatroX. It provides evidence about the research pipelines tested. iatroX's documented reference, reasoning and learning workflows give clinicians practical features to evaluate in their own setting.
Should clinicians choose GraphRAG whenever it is available?
Choose on the demonstrated quality of the complete workflow. Source coverage, applicability, readability, review effort and the selected task matter more than an architecture name alone.
What is a useful first step for a clinician considering iatroX?
Choose a recurring knowledge question, inspect the answer and its sources, then identify whether related question practice or Tutor discussion would help. Keep the reference task and the learning outcome clear when judging usefulness.
Sources and review
Research and product information checked on 4 November 2026. The fictional tests and product-trial framework are original editorial proposals.
- Sesen, Yeung and Asgari: published BMJ CKD RAG and GraphRAG study, 2 November 2026; DOI.
- iatroX methodology, how it works, Tutor, CPD and access.