Plain-English medical summaries are genuinely useful. They make complex clinical discussions accessible, help patients remember what was discussed, and support caregiver involvement in care. But they can become dangerous if users cannot distinguish between three fundamentally different sources of information within the same summary.
The Three-Source Problem
A patient-facing AI summary may contain three types of content, blended together without visible distinction.
What the doctor actually said. Direct transcription-derived content: the diagnosis discussed, the treatment prescribed, the test ordered, the red flags mentioned. This is the most reliable content — though it depends on transcription accuracy, accent recognition, and whether the AI correctly attributed statements to the right speaker.
What the AI inferred. Content the AI generated from context rather than from direct clinician statements: implied recommendations, assumed diagnoses, logical next steps the AI constructed from the clinical narrative. This is where hallucination risk is highest — the AI fills gaps in the conversation with plausible-sounding but potentially incorrect clinical content.
What the medical evidence says. Content the AI added from its medical training data: general health information, medication details, condition explanations, prevalence data, or contextual medical facts. This may be broadly correct but not specific to the patient's clinical context — and may not match what the clinician actually advised. A generic statement about a medication's side effects may be accurate in general but misleading for a patient whose clinician specifically said this side effect was unlikely given their profile.
When these three sources are blended into a single plain-English paragraph — which is how current patient-facing AI scribes present their output — the patient cannot distinguish between "my doctor said this," "the AI thinks my doctor meant this," and "the AI is adding general medical information." The provenance is invisible.
Why Provenance Matters for Patient Safety
If a patient acts on an AI-inferred recommendation that the doctor never made — starting a supplement, stopping a medication, delaying a follow-up, changing a dose — the clinical consequence is real, even though the "recommendation" was AI-generated rather than clinician-authored. The patient believes they are following their doctor's advice. The doctor has no knowledge that the patient received this recommendation. No clinical review occurred.
Kin Health says it builds a health record "grounded in what doctors said." TechCrunch reports that the processing involves transcription, then clinical narrative generation, then user-facing summarisation with action items. Each transformation step introduces the possibility of drift from what was actually said — and without visible provenance, the patient has no way to identify where drift occurred.
What a Safe Patient Summary Should Show
A truly transparent patient summary would distinguish: what was directly transcribed from the clinician's speech (highest confidence — though subject to transcription accuracy), what the AI inferred or synthesised from the conversation (moderate confidence — subject to inference error), and what the AI added from general medical knowledge (variable confidence — not specific to the patient's context).
This level of transparency may be commercially unattractive — it makes the product look less polished and more uncertain. But it is clinically safer — it helps the patient (and any caregiver receiving the shared summary) assess which parts of the information to trust most.
Why iatroX Is Citation-First
iatroX shows clinicians where an answer came from — which guideline, which SmPC section, which evidence source. That citation-first model means the clinician can verify every claim against the original source. Provenance is visible. Verification is possible.
Patient-facing AI summaries would benefit from the same principle: showing the patient what came from their doctor's words versus what the AI added, inferred, or supplemented. Different trust model. Same underlying principle: the user should be able to see where the information came from.