Good clinical notes are relevant evidence for a documentation product, but they do not validate every agent built around that product. Heidi performed favourably in a small simulated ophthalmology scribe study published in March 2026. Heidi II's September launch raises additional questions about selecting tasks, acting on instructions and recovering when circumstances change.
The distinction is not an attempt to discount encouraging research. It is a requirement to match each claim to the capability that was actually evaluated. A note can be accurate while a subsequent action is inappropriate, and a well-executed action can still rest on an incomplete note.
What the ophthalmology study actually assessed
The study, published in Eye on 3 March 2026, compared six scribing tools using seven simulated ophthalmology encounters. Heidi received the highest overall documentation-quality rating among the tools tested. All tools nevertheless produced errors, particularly omissions. This was a small, specialty-specific simulation study of earlier scribing functionality, not an evaluation of Heidi II's newly announced agents. See Zou and colleagues' original study.
Those boundaries are part of the finding, not an optional qualification. The result supports a favourable assessment of documentation performance under the study conditions. It does not provide a general clinical-accuracy percentage, establish superiority in every specialty or show what happens when generated content triggers work in another system.
A fair interpretation therefore holds both points together: the documentation result is encouraging, and errors remained possible. Neither cancelling the favourable result nor turning it into an unrestricted safety claim would reflect the research.
Why downstream action changes the consequences of omission
Consider a fictional encounter in which a clinician discusses a possible referral but agrees to wait until an external assessment is available. A generated note retains the referral discussion but omits the condition that should delay it.
If the note remains a draft, a reviewer may correct the missing condition. If a later workflow uses that note to identify outstanding tasks, the omission could become an incomplete instruction. If execution then proceeds, the practice might initiate work that was not yet agreed. The original documentation defect has changed role: it now affects what the system believes it should do.
This is a hypothetical error chain, not a documented Heidi II incident. Its purpose is to demonstrate why an omission that appears minor during text review may become consequential when another component interprets the text as an operational instruction.
The reverse is possible too. A carefully designed workflow could expose the missing condition by asking for confirmation before proceeding. Whether it does so is precisely what an agent evaluation should test.
Evaluate distinct capabilities separately
Documentation fidelity asks whether the note represents the encounter. Task identification asks whether the system correctly recognises what needs doing. Instruction interpretation asks whether it understands conditions, scope and authority. Execution accuracy asks whether the intended operation occurs in the correct place. Recovery asks whether incomplete or failed work is handed back intelligibly.
Patient outcomes sit beyond those intermediate measures. A system might perform the mechanics well but generate no measurable benefit for the service, or improve a process while creating additional review work elsewhere. Each level needs an appropriate measure rather than one inherited score.
This separation also makes improvement more practical. When a workflow fails, the team can identify whether the problem arose in the source note, task specification, permissions, external interface or completion check. Calling every problem a hallucination obscures the component that needs attention.
What Heidi says about the new agent boundaries
In its 29 September 2026 product introduction, Heidi describes review controls and limits on what agents may do. These are relevant design features for assessing the transition beyond scribing. They should not be conflated with results from the earlier ophthalmology study.
An organisation should obtain the boundaries applicable to its own configuration and test how they behave under realistic uncertainty. A control described at launch is a starting point for evaluation, not a substitute for observing how the whole workflow operates with the organisation's records and connected systems.
The new capabilities are also geographically restricted: Heidi's 28 September announcement excludes the UK and EU. That qualification applies to the announced capabilities, not retrospectively to every earlier Heidi documentation function or study.
Add tests that fluent writing cannot pass on its own
An agent assessment should include contradictory records, missing context and changes made after a task was prepared. For example, an approved test record could contain an old plan, a newer instruction that supersedes it and an attachment whose content conflicts with the summary. The expected behaviour should be defined before the run.
Another scenario could begin with a clear instruction but interrupt the workflow after an external system accepts part of it. The assessment would examine whether the agent distinguishes preparation from submission and submission from confirmed completion. A polished final message should not receive full credit when the operational state remains uncertain.
A third scenario could change the plan while related work is pending. Reviewers would inspect whether the change reaches every relevant task and whether already completed actions are described accurately. These are proposed tests; we have not run them against Heidi II for this article.
Make the denominator visible
A report of successful tasks needs to identify which tasks were eligible, which were attempted and which were excluded. If a system appropriately declines an unsupported request, that may demonstrate a useful boundary. If it silently omits a difficult subtask while reporting success, that is a different outcome.
We would report complete success, partial completion, appropriate escalation and incorrect action separately. Staff rescue effort should remain visible even where the final patient-facing result is correct. Otherwise the report may credit the software for work completed by people after the automation stopped.
Independent review should also examine whether the task itself was appropriate. Efficiently carrying out an unnecessary action is not the same achievement as reliably completing agreed work. The benchmark must therefore include interpretation and authority, not just interface navigation.
A balanced conclusion for clinical buyers
For a clinician choosing a documentation aid, scoped scribe research is directly relevant. For an organisation considering agentic execution, it is one part of a wider evidence package. For an educator, the new question is whether users understand the case well enough to recognise when apparently faithful documentation has left something important out.
The appropriate response is not to discard the earlier evidence. It is to build on it without extending its conclusions beyond the tested task. iatroX's own educational and reference outputs deserve the same discipline: published methods and reviewed content describe safeguards and design, not an assurance that every answer or learning interaction is correct.
Frequently asked questions
Did the ophthalmology study prove Heidi II is safe?
No, the March 2026 study assessed earlier scribing functionality using simulated ophthalmology encounters. It did not evaluate the agents announced in September 2026.
Was Heidi the best-performing scribe in that study?
Heidi received the highest overall documentation-quality rating among the six tools tested. That scoped result should not be presented as universal superiority or a clinical-accuracy percentage.
What should an agent evaluation add to note-quality testing?
It should examine task selection, conditional instructions, execution, exceptions and recovery, with transparent denominators. Relevant patient outcomes and remaining human work require separate measurement.