An NHS organisation should consider pausing or restricting an AI tool when a credible safety signal calls its current controls into question, particularly where further use could expose patients to avoidable harm. The response should match the affected function and uncertainty. Neither a single incident count nor strong adoption is enough to settle the decision.
This is a proposed operational framework for England, dated 10 October 2026, not a statutory decision rule. GMC guidance on acting on concerns requires those with relevant responsibilities to respond to patient-safety concerns. The practical challenge is translating that obligation into authority, continuity and a testable route back to use.
Define what has become uncertain
A fictional hospital identifies that an AI-generated discharge summary sometimes places a historical medication change among current instructions. The first question is not whether the entire supplier should be banned. It is what function failed, which records may be affected and whether staff can reliably detect the problem before it influences care.
The signal might originate from a patient, a clinician, a pharmacist, an audit or the supplier. Its source should not determine whether it deserves assessment. A complaint containing a specific, plausible account can be important even before the organisation knows how frequently the problem occurs.
Separate confirmed facts from unresolved questions. The team may know that a particular output was wrong, while not yet knowing whether the cause lies in the input, configuration, integration, model or review process. That uncertainty can justify temporary controls without requiring an immediate final explanation.
Distinguish restriction, pause and withdrawal
Restriction narrows permitted use. A service might disable automatic transfer while allowing draft generation in a supervised test environment. A pause stops the affected live function while the organisation investigates. Withdrawal ends use because the risk, value or support arrangements no longer justify continuing.
These are proposed operational distinctions, not universal legal categories. Their usefulness is that they avoid an all-or-nothing debate when the product performs several different jobs. Restricting one consequential function may be more appropriate than suspending unrelated functions whose evidence and controls remain intact.
However, narrowing use must be technically and operationally credible. A written instruction not to press a button is weak if staff cannot tell when the system will act automatically. Where the boundary cannot be enforced, a broader pause may be necessary.
Use a decision sequence that can be followed under pressure
The following sequence is intended for adaptation by clinical, technical and governance leads. It should be rehearsed before an incident, not improvised while teams argue over who can stop the system.
| Decision point | Proposed response |
|---|---|
| Could continuing use cause serious harm before review? | Stop the affected function and arrange immediate clinical assessment |
| Is the scope uncertain but a consequential failure credible? | Apply temporary restrictions while identifying affected activity |
| Is the problem contained by a demonstrably effective control? | Consider bounded continuation with enhanced monitoring and a named decision-maker |
| Has the issue invalidated a core assumption of deployment? | Reassess the safety case and operating procedure before expansion or restart |
| Can the supplier or organisation no longer support safe use? | Plan withdrawal and continuity rather than indefinite informal workarounds |
The assessment should consider severity, likelihood, exposure, detectability and reversibility. These are dimensions for judgement, not a formula that generates permission to continue. An uncommon but hard-to-detect consequential error can matter more than numerous minor formatting corrections.
Keep care running while stopping the software
A pause is not complete when access is disabled. In the fictional discharge scenario, staff need to know which summaries are still drafts, which have been released and which patients may need review. The usual discharge process must remain available, including a way to manage the additional work.
Someone should reconcile pending actions. Otherwise a well-intentioned stop can create a second problem: unfinished referrals, unsent messages or duplicate tasks when staff repeat work they cannot see. A concise handover should identify the original objective, completed steps, unresolved items and current owner.
DCB0160's deployment and lifecycle scope, checked on 10 October 2026, makes ongoing operation and eventual decommissioning relevant to clinical-risk management. A fallback should therefore be part of the deployment design, not merely an emergency workaround discovered after failure.
Investigate without losing the evidence
Preserve relevant records through approved organisational channels. The investigation may need the original input, generated output, product version, configuration, user actions and destination-system response. It should not depend on a clinician copying patient material into a personal account.
The purpose is to distinguish plausible causes and identify effective controls. A review that concludes only "human error" because a clinician signed the output may miss whether the interface, workload or integration made meaningful checking difficult. Equally, labelling every failure "the AI" may obscure an incorrect source record or local configuration.
HSSIB's investigation into implementation of the Patient Safety Incident Response Framework, published in October 2025, is useful background for learning-focused responses. It does not prescribe a unique AI stop-use threshold. The local investigation should remain proportionate to the safety questions it must answer.
Make reinstatement harder than receiving a reassuring email
A supplier's explanation is relevant evidence, not by itself a complete restart decision. Specify what changed, which failure mechanism it addresses and how the organisation tested the revised workflow. A model update may correct one behaviour while altering another.
For the fictional discharge tool, a restart test could include synthetic histories with multiple changes over time, contradictory source entries and incomplete information. Assess the whole path into the approved record, not just a screenshot of the corrected draft. These are proposed tests, not results from an actual run.
Agree a bounded restart, monitoring arrangements and a fresh review point where appropriate. Communicate what staff should do differently and what remains prohibited. If reviewers cannot explain the new controls, the organisation has not yet demonstrated a usable change.
Do not make either perfection or popularity the threshold
Manual services also make mistakes. A fair decision compares the assisted workflow with realistic alternatives, including the risks of delay, increased workload or reduced access during suspension. That does not justify ignoring an unresolved hazard; it ensures that the response protects care rather than merely removing a visible technology.
The same principle applies to source-linked reference tools, including iatroX. A concerning answer should be challenged and reported, not accepted because citations are present. Professional learning can examine what failed and what needs checking without using an unrelated tool as the repository for incident evidence.
A credible stop-use policy preserves the ability to question, restrict and recover. Its success is not measured by never stopping a product, but by making justified decisions before uncertainty becomes avoidable harm.
Frequently asked questions
Should one AI error automatically trigger withdrawal?
Not automatically; the nature, consequences, exposure and effectiveness of controls matter. A single serious signal can nevertheless justify an immediate pause while the organisation establishes the scope.
Who should have authority to stop an AI tool?
The organisation should name an accountable decision-maker and an urgent route for staff to obtain restrictions. Technical access to disable a system and authority to decide on clinical use should be coordinated.
What evidence should be required before restarting?
The organisation should understand the change, test the affected workflow and confirm that staff can use the controls. A supplier assurance without local verification may leave the original deployment assumptions unresolved.
Use a safety question to guide focused professional learning →