An AI system can affect care without naming a diagnosis. It may influence who gets an appointment, where a referral goes, whether a patient receives a message or which unfinished task attracts attention. The safety question is what the system changes in the patient's journey, not whether its interface describes the work as administrative.
This analysis is dated 10 October 2026 and focuses on NHS services in England. The MHRA's clarification of 29 July 2026 distinguishes documentation-only ambient functions from functions supporting clinical decisions or certain automated actions in Great Britain. Classification must be assessed for the actual function; being outside medical-device regulation does not establish an absence of clinical risk.
Follow one patient through several ordinary decisions
Consider a wholly fictional patient seeking review of a persistent problem. They submit an online request while at work, answer briefly because the form is difficult to use and mention an accessibility need in a free-text field. An automated system places the request into a routine queue.
A staff member subsequently reviews it and arranges appropriate care. After the consultation, another system prepares a referral. The referral is sent but rejected because required information is missing. A later reminder is issued through a channel the patient cannot reliably access, and an administrative status eventually marks the process as complete.
No AI in this constructed example independently diagnoses the patient. Yet several decisions could determine whether the person reaches the next stage of care. The scenario is not an allegation about a named NHS service or commercial product. It is a way to examine the interfaces that an evaluation confined to diagnostic accuracy would miss.
Appointment allocation is an access decision
The first question is whether the system understands enough to allocate the request appropriately. A brief answer may reflect the patient's communication circumstances rather than a straightforward problem. An unanswered field may mean the question was not understood, not that the relevant feature is absent.
For a proposed evaluation, review requests that were completed, abandoned and redirected. Examine what happens when a patient cannot describe the issue in the expected way. Include an accessible human route rather than measuring only the speed at which confident digital users reach an appointment.
HSSIB's July 2024 investigation of online consultation tools identified implementation and patient-safety issues in an earlier generation of digital access services. It did not test the fictional system described here or establish how current AI will perform. Its relevance is the need to assess the access pathway, not just the submitted form.
A referral has more states than sent and not sent
A useful referral workflow distinguishes preparation, authorisation, transmission, acceptance, rejection and completion of the intended next step. These are proposed operational categories, not a claim that every clinical system uses those exact labels.
In the fictional journey, a technically successful transmission is followed by rejection. Someone must receive that information, understand its significance and arrange the next action. The patient should not be left to discover the problem when an expected appointment fails to arrive.
The HSSIB interim report on electronic referrals, published on 18 August 2026, provides a current primary source on ongoing-care handovers across electronic systems. It should be read as evidence about referral processes, not repurposed as a finding that generative AI caused the problems under discussion.
A local AI test should therefore continue beyond the first successful interface response. Simulate a rejection, a changed destination and a request returned to an absent clinician. Check whether the receiving team can identify the unresolved responsibility without reconstructing the whole transaction.
Prioritisation can hide work as well as surface it
A system that ranks a waiting list or inbox should be examined for inappropriate exclusion as well as inappropriate inclusion. A plausible-looking priority list tells the reviewer little about people who never appear on it.
For the fictional patient, an unresolved task might be deprioritised because a message was already sent. That is a potential design failure: communication activity has been treated as resolution. Another system might repeatedly escalate the same low-value task, consuming attention that should reach genuinely unresolved care.
The proposed safeguard is to inspect the selection rule and the omitted cases. What information drives prioritisation? How is missingness handled? Can a reviewer see why a task has disappeared from the active queue? A review of only the highest-ranked items cannot answer those questions.
Accessible communication is part of completion
A delivered message does not prove that the patient received, understood or could act on it. The service needs a proportionate response to non-response, delivery failure and requests for another format.
In the fictional scenario, the patient has not declined care. The communication channel does not meet their needs. An automated status should not convert that difficulty into a conclusion about motivation or adherence. The record should preserve what is known: the message was sent, a response was not received and the accessibility issue remains relevant.
A useful service review asks whether staff can change the channel, reach a person and reopen a task without fighting the software. The safeguard is not endless automated messaging. It is an intelligible route for recognising that the standard process has not worked.
Build a service-level safety map
The following proposed map focuses on the points at which a superficially administrative output could affect care.
| Transition | Question for the service | Evidence to inspect |
|---|---|---|
| Request to queue | Was sufficient information available for routing? | Missing fields, redirections and abandoned requests |
| Consultation to referral | Did the authorised intention survive drafting? | Approved content, urgency and destination |
| Referral to receiving service | Was it accepted and assigned? | Rejection notices and unresolved handovers |
| Plan to patient message | Could the patient understand and act? | Delivery failures, accessibility and clarification requests |
| Reminder to closure | What established that work was resolved? | Completion evidence or a documented alternative outcome |
For each transition, name a person or team able to resolve exceptions. A supplier support desk may investigate a technical failure; it is not automatically responsible for the patient's clinical follow-up. Keep those roles distinct.
Use clinical knowledge without confusing it with workflow ownership
A clinician may need to revisit the evidence when deciding whether an administrative interruption changes the clinical plan. As described in iatroX's product explanation checked on 10 October 2026, its reference and learning functions support knowledge work. They are not an NHS inbox, referral owner or substitute for local task tracking.
A useful learning exercise is to review the fictional journey and identify the first point at which someone should have recognised that the usual process was no longer adequate. Then ask what evidence, authority and communication that person would need.
The central lesson is not that administration should remain manual. It is that automation should be evaluated against completed, appropriate care, including the patients and tasks that fail to follow the expected route.
Frequently asked questions
Can administrative AI create patient-safety risks without diagnosing illness?
Yes, if its outputs affect access, routing, communication or completion of care. The risk depends on the function and surrounding workflow rather than the administrative label alone.
Does being outside medical-device regulation mean a tool needs no local assessment?
No. Regulatory classification and the safety of a particular deployment answer different questions, and organisations still need to assess the consequences of use.
What should count as a completed automated referral?
The service should define the endpoint relevant to its responsibility, including acceptance and unresolved follow-up. A successful transmission alone should not be treated as proof that the intended care occurred.
Practise identifying the clinical question behind a workflow problem →