skip to main content
iatroX JournalClinical AI

What Should "Provenance" Mean in Clinical AI? Four Different Questions

Featured image for What Should "Provenance" Mean in Clinical AI? Four Different Questions

Provenance in clinical AI should answer more than "which reference was cited?" It should help establish where the medical claim came from, where the patient information came from, which software produced the output and who reviewed or acted on it. These are related questions, but a source link answers only part of them.

One word covers several kinds of origin

The W3C PROV overview, published on 30 April 2013, describes provenance through the entities, activities and people involved in producing information. In healthcare, that general concept can help distinguish source material from the transformations and decisions built on it.

The HL7 FHIR Provenance specification, inspected on 10 October 2026, provides a healthcare interoperability resource for describing how information was created or changed and the agents involved. It is not a certificate that the resulting clinical interpretation is true.

A practical clinical-AI specification therefore needs both traceability and limits. It should reveal the chain sufficiently for appropriate review without implying that an authentic chain guarantees a correct conclusion.

Where did the medical claim come from?

Suppose an AI answer recommends a particular approach and supplies a guideline link. The user should be able to identify the relevant recommendation, not merely reach a long document with a matching topic.

Useful source provenance includes the document, relevant passage, version or date and the relationship to the claim. Does the source directly support the statement? Is the answer synthesising several sources? Does the recommendation concern the requested population and setting?

These are proposed review requirements. They go beyond checking whether a citation exists, because a genuine citation can accompany a misleading paraphrase or an inapplicable recommendation.

If a source later changes, the provenance should also make it possible to distinguish what was available when the answer was produced from what the current page now says. A live link alone may not preserve that distinction.

Where did the patient information come from?

The next layer concerns the facts used to make the answer patient-specific. A diagnosis might come from a current clinical assessment, a historical letter, a patient report or an AI-generated summary. Those origins carry different meanings.

Consider a fictional referral in which a tentative diagnosis is copied into a summary and then used as a confirmed premise in a later recommendation. The recommendation may cite appropriate medical guidance. Its weakness lies in the status of the patient information, not the reference library.

A useful record would preserve who made the original statement, when it applied and whether it was reported, observed, suspected or confirmed. It should also show when later information supersedes it.

This does not mean every sentence needs an elaborate annotation visible during a consultation. It means that consequential facts should have inspectable origins and that the interface should not erase distinctions the clinician needs to judge applicability.

Which software produced this output?

A product name is not necessarily enough to identify the system that generated an answer. The relevant configuration may include the model, retrieval collection, instructions, enabled tools and interface behaviour.

A proposed evidence record should therefore identify the available product or model version and material configuration at the time of use. Where the supplier does not expose a component, that limitation should be recorded rather than replaced with a guess.

The purpose is practical. If a later update changes behaviour, the organisation needs to know which outputs and evaluations relate to the earlier system. It should not assume that a favourable study of one configuration automatically applies to every later release carrying the same brand.

Exact reproducibility may remain difficult even with good records. Provenance can support investigation without promising that the identical answer can always be regenerated from a saved prompt.

Who reviewed, changed and acted on the result?

A generated draft, a clinician-approved plan and an action completed in another system are different objects. Provenance should make their relationship visible rather than collapse them into a status such as "reviewed".

In a fictional workflow, an AI drafts a referral, a clinician changes its urgency, and an administrator submits the revised document. A useful trace would distinguish the draft from the approved version and the submission from confirmation that the receiving service accepted it.

The clinical importance is that each stage answers a different question. Approval may establish that a named person accepted a particular version. It does not establish what they inspected or whether a downstream action succeeded.

Organisations should define these statuses before relying on them in audit. Otherwise, a detailed activity log may create false reassurance while leaving the consequential transition unclear.

A minimum specification for discussion

The following is a proposed procurement and design specification, not a mandatory FHIR profile or a claim that every product currently implements it.

LayerMinimum useful informationWhat it must not imply
Medical evidenceIdentifiable source, relevant passage and date or versionThat citation alone proves applicability
Patient contextOrigin, subject, timing and status of important factsThat every recorded statement is confirmed
SoftwareAvailable version and material configurationThat a brand name identifies every component
Human and operational actionReviewer, approved version, action status and destinationThat approval proves comprehension or completion

The specification should be tested with a realistic task. Ask a reviewer to trace one consequential conclusion backwards and one correction forwards. Can they find the originating fact? Can they identify which later outputs or actions relied on it?

An attractive provenance panel is not useful if it cannot answer those questions without specialist technical assistance during the relevant workflow.

Traceability should not become indiscriminate retention

Keeping every raw conversation forever is not the only way to support accountability. The organisation should decide which evidence is necessary for each purpose and use appropriate access and retention controls.

A source identifier, approved output, version record and focused event log may answer some questions without duplicating a complete consultation in several systems. Other investigations may require additional evidence under an agreed preservation process. The appropriate design depends on the workflow and its obligations.

This is a governance decision to make with clinical, technical and information-governance input, not something an individual clinician should improvise by saving patient material in a personal account.

Make provenance useful to the person checking the answer

As described in iatroX's October 2026 product information, Ask-iatroX provides linked clinical sources. That addresses a useful part of medical-claim provenance. It should not be presented as a complete record of patient-data lineage, local approvals or actions in another clinical system.

For a reference question, the reader may primarily need the source and its applicability. For an integrated care workflow, the additional patient, software and action layers become important. The necessary depth follows the task.

The standard worth pursuing is not the largest audit trail. It is the ability to understand where a consequential statement came from, what changed it and what depended on it. Provenance makes investigation and correction possible; clinical assessment determines whether the resulting claim should be used.

Add an assurance record without losing the four-layer framework

The four questions above describe the origins of clinical claims, patient information, software outputs and subsequent actions. An additional assurance record can connect those layers to the evidence supporting the deployed configuration. This extension was added on 13 October 2026; it does not replace the four-layer framework or turn certification into another clinical reference.

The MHRA's intended-purpose guidance makes the relationship between the product's claimed use and supporting evidence important. The joint transparency principles also support communicating information relevant to use, limitations and changes. Together, they provide a foundation for asking whether the evidence a buyer is shown actually concerns the system being used.

A clinical citation answers a question about the support for a medical proposition. An assessment document answers a different question about a product or management system within its scope. Neither can be used as a substitute for the other.

Proposed assurance-record fieldQuestion it should resolve
Product and legal manufacturerWhich system and responsible entity does the evidence concern?
Configuration referenceWhich release, material components and permissions were evaluated?
Intended purposeWhich task, users, population and environment are supported?
Evaluation referencesWhich findings support which claims, and what limitations remain?
Applicable assessment documentsWhat was assessed, under which framework and scope?
Local decision and restrictionsWhich configuration is permitted here, and under what conditions?
Change linkageWhat changed since evaluation, and how was its impact assessed?

This is a proposed traceability record, not an official certification register or a requirement to publish confidential technical material openly. Its purpose is to make the connection inspectable by the people who need it.

A sound reference can coexist with an unsupported product claim

Consider a fictional outpatient tool that produces a follow-up explanation. The linked guideline is authentic, current and accurately represented. The patient information also supports that explanation. However, the evidence supplied to the hospital concerns a draft-only configuration with clinician review, while the proposed new configuration sends the explanation directly to patients.

The clinical citation has not become false. It simply does not establish whether the new delivery function, review boundary and exception handling are adequately supported. An assurance record would expose that the cited product evaluation and the deployed configuration answer different questions.

The next step is to assess the changed function and its evidence, not declare the entire product unsafe solely because the configurations differ. The joint change-control principles support bounded, evidence-based consideration of modifications. Depending on the route and change, further assessment may be needed before the claim is extended.

Test the record in both directions

A proposed audit should start with one live configuration and trace backwards to its applicable evaluation and assessment documentation. It should then start with a changed or superseded evidence item and identify which deployed configurations and claims depend on it.

The exercise should reveal missing links, not manufacture precision. If an external model version is not available, record that limitation and its implications. If an evaluation covers only a narrower use, preserve that boundary rather than attaching the study to the whole brand.

Source-linked reference tools illustrate only one part of this chain. As described on 13 October 2026, Ask-iatroX supports inspection of clinical sources; that is not a complete product-level assurance record for another system. The stronger standard is to connect what was assessed with what is being used, while retaining the separate question of whether the individual clinical statement is appropriate.

Frequently asked questions

Is provenance the same as a list of citations?

No. Citations address medical sources, while provenance can also describe patient-data origins, software transformations, review and subsequent actions.

Does FHIR Provenance prove that an AI output is clinically correct?

No. It supports representing origins and changes, not validating the medical truth or appropriateness of the resulting interpretation.

Should every clinical AI interaction be retained indefinitely?

No universal retention period follows from the need for provenance. Organisations should define necessary evidence, access and retention by purpose and applicable requirements.

Examine the clinical sources behind an answer →

More from the Journal