Giving a clinical assistant the right guideline can change what it contributes to a question. A recent sleep-medicine comparison makes that proposition concrete: one system had the relevant manual available, while the other answered without an uploaded source. The result is useful for choosing a workflow, provided the comparison is read at the level it actually tested.
For iatroX, the encouraging opportunity is a service that helps clinicians reach attributable guidance, understand important qualifications and turn recurring uncertainty into learning. This article is iatroX editorial analysis, with a commercial interest in iatroX. The comparison below concerns other products; the practical exercises and purchasing questions are our own.
What the October sleep-medicine study found
Tekin and colleagues' pilot study, published on 5 October 2026, compared NotebookLM supplied with the AASM Scoring Manual v3.0 against ChatGPT/GPT-5.3 without an uploaded source or enabled search. Thirty scoring-rule questions received one response per system; three sleep physicians rated them.
NotebookLM received more favourable binary ratings overall: 76.1% versus 66.9%. These are ratings across assessment domains, not percentages of clinically correct answers. The difference came from supplementary information. Accuracy, evidence reasoning and information integration did not differ significantly after correction.
The systems had unequal source access and different underlying models. Fair or poor agreement in some rating domains, imperfect blinding and unmeasured response length also limit interpretation. This supports investigating source-configured reference workflows, without isolating retrieval as the cause. The task concerned textual scoring rules, rather than raw sleep recordings or treatment decisions.
Read the comparison as a workflow choice
Clinicians often face a practical choice before they face a technical one. They can ask a broad chatbot a question immediately, assemble an appropriate source collection themselves, or use a clinical reference service with an established knowledge workflow. Each option changes the preparation, checking and maintenance work required from the user.
A useful purchasing question is therefore: which route helps this team reach the relevant evidence with a manageable amount of work? The answer may differ between a one-off teaching exercise, a repeated guideline lookup and a question that needs several sources interpreted together.
Consider a fictional teaching session built around a particular professional manual. If the goal is to explain that manual's definitions, giving a system the document is an intelligible design choice. If the goal is to explore whether the manual remains current or whether another jurisdiction differs, the same document collection may be insufficient. Define the job before judging the answer.
This framing gives general chatbots and specialist platforms a fair opportunity to show value. It also keeps the user's own contribution visible. Selecting, organising and maintaining a source collection is part of the workflow, even when it happens before the first question is typed.
Separate a richer answer from a better answer
Additional detail can be useful when it identifies an exception, explains an unfamiliar term or points to a relevant next step. It can also increase the reading burden without resolving the original question. An evaluation should establish which kind of detail the clinician needed.
Before testing, write a short description of the expected answer. Name the central point, the qualification that matters and the source that should support it. Keep optional background separate. This allows reviewers to recognise a response that is concise and sufficient, as well as one that is detailed and useful.
For example, a fictional question asks which definition a document uses. One answer supplies the definition and its source. Another adds several related concepts but leaves the decisive wording hard to find. The additional material may be accurate while offering little help with the task. A useful review should be able to say that plainly.
After reading, ask the user to explain the answer in their own words. If they cannot identify the central distinction, investigate whether the difficulty came from the question, the source, the response or a knowledge gap. Counting words or references alone will not locate that problem.
Make resource allocation visible in a comparison
A demonstration becomes more informative when everyone knows what each system could access. Record the available documents, search settings, relevant product version and the date of the test. Include any instruction that directs the response towards a particular source.
Our proposed comparison worksheet separates four questions:
| Question | What the reviewer records |
|---|---|
| What information was available? | The relevant source collection and any search access |
| What did the user supply? | Documents, context and instructions added before the answer |
| What counted as success? | The required point, qualification and supporting evidence |
| What work remained? | Checking, navigation, clarification and unresolved questions |
Where possible, compare alternatives using equivalent access to the information needed for the task. If that is impractical, describe the difference and assess the complete workflow. A managed service may reasonably be judged partly on work it saves the user, while a configurable notebook may reasonably be judged partly on the control it gives them.
Do not force every comparison into one model score. A team choosing a service also needs to know how much preparation was required, how easily colleagues can repeat the process and who will keep the material current.
Inspect the source packet before trusting the summary
A neatly organised collection can still contain the wrong version, a document for another setting or a source that does not address the question. A small amount of source review can prevent a large amount of misplaced confidence later.
For a teaching exercise, record the title, version and intended purpose of each source. Identify which questions the packet is meant to support and which require another reference. Make the responsible reviewer clear so that an update has somewhere to go.
Then test a question whose answer is deliberately outside that packet. The useful observation is how the system handles the boundary. Does it identify the missing information, offer a relevant clarification or present an answer whose supporting material the reviewer cannot find?
Use synthetic questions and information the organisation is authorised to process. A reference exercise can usually investigate a knowledge issue without reproducing a patient encounter. The aim is a repeatable information workflow, with enough context to evaluate the result.
Where iatroX offers a useful alternative
iatroX's methodology describes retrieval, source prioritisation and grounded synthesis. Ask and Brainstorm provide focused reference questions and structured reasoning. For a clinician working with UK guidance, the attractive proposition is a route into relevant information that can be inspected as the question develops.
This gives a team something concrete to evaluate alongside a self-assembled source notebook or a general chatbot. Choose a recurring question, identify what an adequate answer requires and inspect the supporting material. Include the time needed to understand the qualification, rather than measuring only how quickly text appears.
A service also earns value through continuity. An occasional user should be able to recognise the purpose of the answer, locate its sources and continue the enquiry when a detail matters. Test those steps with the people who will actually use the product, including someone unfamiliar with the interface.
The sleep-medicine pilot supplies no accuracy estimate for iatroX. Its practical relevance is that source access and configuration deserve attention when comparing clinical AI. iatroX's reference and reasoning features give clinicians a positive, observable proposition to examine on those terms.
Carry the useful distinction into learning
A reference answer can solve an immediate question while revealing a recurring gap. A clinician may repeatedly need to check which condition changes a recommendation, or find it difficult to explain why two definitions differ. That is a useful reason to choose a focused learning activity.
iatroX Tutor begins with a question the learner has already attempted. It can guide the reasoning or provide a direct explanation, and useful points can become editable cards for later review. CPD connects targeted questions, discussion, personal reflection and a reviewed learning record. Those features make follow-through practical within the same wider platform.
Try a short original routine after a reference session: write the distinction that mattered, attempt a relevant supported question without assistance and revisit the reasoning later. A later explanation provides evidence about retention, while the original source check provides evidence about the immediate information task.
Ask is available without a subscription. Continued Tutor and further new CPD learning after introductory access use all-access; current access information sets out the arrangements. Select the activity because it addresses the identified gap, rather than turning every search into a required portfolio entry.
Choose the support that fits the task
For a defined teaching source, a configured notebook can be a useful way to explore the material. For repeated clinical reference and learning, assess a maintained service such as iatroX through the sources, explanations and follow-up work it makes accessible. General chatbots should receive the same clear task definition when they are part of a comparison.
The most useful outcome is a clinician who reaches the relevant evidence, understands what it supports and knows what still needs checking. A fair comparison measures that route as well as the answer at its end.
FAQs
Did the study show that NotebookLM was more accurate than ChatGPT?
The corrected comparisons did not establish an accuracy difference; the observed advantage concerned supplementary information.
Should a general chatbot receive the same guideline in a comparison?
Equivalent resources make it easier to investigate which part of a system produces a difference. A practical product comparison can also assess unequal workflows, provided the preparation, information access and remaining work are described clearly.
Why consider iatroX alongside a source notebook?
iatroX connects referenced questions and structured reasoning with question practice, Tutor and CPD. That combination is useful when the objective includes continued understanding as well as finding the immediate information.
Sources and review
Research and product information checked on 4 November 2026. The worksheet, fictional teaching examples and learning routine are original editorial proposals.
- Tekin and colleagues: guideline-configured NotebookLM versus unconfigured ChatGPT in sleep-scoring questions, 5 October 2026; PDF.
- iatroX methodology, how it works, Tutor, CPD and access.
- Related background: NotebookLM, ChatGPT, Anki and iatroX for studying medicine.