When an AI answer may be unsafe, address the immediate clinical risk before investigating the software. Preserve relevant information securely, record what was expected and what occurred, and use the appropriate local and supplier reporting routes. Potential harm, confirmed harm and an educational error are different situations, but none is made irrelevant by the answer's confident wording.
The patient comes before the screenshot
A clinician who identifies a potentially harmful recommendation should not delay necessary assessment or care while trying to reproduce the output. Use the appropriate clinical support and local process for the situation.
The GMC's Good medical practice guidance, checked on 19 September 2026, addresses acting on safety concerns, secure records and reporting incidents involving medical devices, including software and digital tools. It also recognises potential risk, not only harm that has already occurred.
The immediate question is what needs correcting or containing now. That may concern a draft that has not been used, information already communicated or an action that requires review. The incident investigation follows alongside, not instead of, the clinical response.
Describe the failure without guessing its cause
Consider a wholly fictional reference query. A clinician asks about a general management distinction. The answer makes a recommendation, but the linked source concerns a different population. The clinician notices the mismatch before acting.
A useful report describes that observable discrepancy. It does not need to diagnose the underlying model failure or claim that the supplier deliberately misrepresented evidence.
Separate the task, input, output, source and consequence. The task was the intended question. The input was what the system received. The output was the statement produced. The source was the material used or cited. The consequence was whether anything happened because of the statement.
That separation allows a supplier or investigator to distinguish retrieval, interpretation, interface and workflow problems. "The AI hallucinated" may be understandable shorthand, but it is not enough detail to investigate the event.
Preserve enough information to reproduce the problem
Record the product and relevant function, approximate time, version or model identifier where available, and the sequence of actions. Keep the question, answer and relevant source location in an approved environment where it is lawful and appropriate to do so.
Do not put identifiable patient information into a public issue tracker, social-media post or unapproved email. A screenshot can contain names, dates, tabs or other details beyond the text you intended to share.
Where version information is unavailable, say that. Where the output cannot be reproduced, record the original observation rather than repeatedly changing the question until a more dramatic error appears. A later successful response does not prove the earlier one did not occur.
The record should be sufficient for review without becoming an unnecessary copy of the whole clinical record.
A practical incident description
For the fictional example, an educational report could read:
"During a general reference task, the system recommended an approach supported by a citation that addressed a different population. The discrepancy was identified before clinical use. The original query, output and source passage have been retained securely. The report concerns a possible source-applicability error; no patient harm has been established."
This is original sample wording, not a report of an actual iatroX or competitor incident. Replace its details only with observed facts. Do not claim that no harm occurred if the downstream use has not yet been checked.
Include the expected behaviour and why it matters. For example, the system should have preserved the source's population restriction or identified that the evidence did not answer the question. That makes the concern more actionable than a general statement that the answer was poor.
Use the appropriate reporting routes
Local clinical governance remains important when the tool is used in an organisation. A supplier report can support technical investigation, while local reporting addresses how the event affected the service and whether others may be exposed to the same problem.
The MHRA Yellow Card service includes reporting routes for medical-device software, apps and AI. Its applicability depends on the product and event; a clinical concern should not be dismissed merely because the product is software rather than physical equipment.
Not every inaccurate educational response is a reportable device incident. A wrong practice-question explanation may need a content correction, while a clinical software issue with potential harm may require additional routes. Use the relevant guidance and organisational support rather than treating one form as suitable for every situation.
A report to the supplier does not automatically complete the organisation's responsibilities. Equally, submitting locally does not guarantee the supplier has the information needed to investigate.
Ask what happens after the report
Retain an incident or support reference where provided. Clarify whether further information is needed, how any immediate restriction is communicated and how the organisation will know whether the issue has been addressed.
Do not assume that a corrected answer means the whole failure class has been resolved. A supplier may need to investigate similar questions, source handling or a particular product version. The appropriate scope depends on the actual issue.
For internal learning, discuss the conditions that allowed the problem to reach the user and the checks that detected it. Avoid a blame-focused account that discourages people from reporting future concerns.
Reporting an issue with iatroX
The iatroX clinical AI standards page, checked on 19 September 2026, describes user reporting of suspected errors and missing nuance as part of its feedback process. The published support page also provides hello@iatrox.com as a contact route.
Use the available feedback or support route to provide the relevant page, time and issue description. Do not send passwords, reset links or unnecessary patient information. These public pages do not establish an emergency response service or a guaranteed clinical incident-resolution time.
The same reporting standard should apply to iatroX and other suppliers. Source-grounding and checking processes are design features, not reasons to assume an error cannot occur. A trustworthy product should make scrutiny possible rather than ask users to replace it with confidence in the brand.
Frequently asked questions
Should I wait for the supplier before addressing a possible clinical risk?
No: use the appropriate clinical and organisational response immediately. Technical investigation must not delay necessary patient care.
Can I report an incident if no harm has occurred?
Potential risk can still be important and may fall within relevant reporting requirements. Describe what is known without inventing a harmful outcome.
Should I post the full conversation publicly to prove the error?
No: preserve evidence securely and use appropriate reporting routes. Public disclosure can expose patient information and omit context needed for a fair investigation.
Discuss clinical AI safety and evaluation with iatroX Insights →