skip to main content
iatroX JournalClinical AI

Heidi Work's Success Might Mean Fewer Logins, Not More

Featured image for Heidi Work's Success Might Mean Fewer Logins, Not More

An autonomous clinical agent could become more valuable while its users spend less time interacting with it. It could also receive fewer logins because people have stopped using it. For Heidi Work, interface engagement without evidence of delegated work would leave those opposite explanations unresolved.

Heidi's task-handling preview, checked on 28 September 2026, excludes the UK and Europe. This article evaluates the measurement problem under a working premise of substantial authorised delegation. The examples and proposed metrics are analytical designs, not Heidi Work usage data, confirmed interface functions or claims about its business model.

An agent's interface may not be where its value appears

A drafting assistant can be useful because a clinician repeatedly opens it, reviews the output and moves that output into another system. In that arrangement, interaction is closely connected to the work being done.

An authorised agent presents a different possibility. The clinician could set a permitted workflow in motion and then spend most of their time elsewhere. Much of the value might appear in a completed action, an updated record or an appropriately resolved exception rather than in a long session inside the product.

This does not make engagement metrics obsolete. A sudden increase in screen time might reveal onboarding, uncertainty or repeated corrections. A sudden decrease might indicate that a workflow has become dependable, that eligible activity has declined or that the product has been abandoned. The metric becomes a prompt for investigation rather than an outcome that is inherently good or bad.

The same logic applies to notifications. Fewer alerts could mean fewer problems, poor visibility or a failure to notify. More alerts could mean useful escalation or a system repeatedly interrupting people with information they cannot act on. Counts need to be interpreted alongside their purpose.

Separate access, activation and repeated delegation

A licence establishes availability. A connected account suggests that setup has progressed. A completed demonstration shows that a particular pathway worked under those conditions. Repeated delegation shows something different again: a user is choosing the workflow for suitable work over time.

An adoption report should preserve those distinctions. It should not treat every licensed clinician as an active delegator or every activated account as evidence of routine use. It should also distinguish individual behaviour from practice-wide adoption. One person can generate substantial activity while colleagues remain unable to see or rely on the resulting work.

The denominator matters even before the first task. A clinician with few relevant administrative workflows has a different opportunity to use the agent from a clinician whose sessions repeatedly generate them. Raw task counts would make those users look more different than their actual willingness to delegate might be.

A scribe trial illustrates the access-versus-use problem

The UCLA randomised trial published in NEJM AI on 26 November 2025 involved 238 physicians. The two assigned ambient scribes were used in approximately 30% to 34% of visits within their respective intervention groups. The trial therefore illustrates why providing access should not be equated with use in every encounter.

Those figures describe the study's scribe interventions, not autonomous agents or Heidi Work. They are not a target adoption rate for another product, and selective use may be appropriate. The transferable measurement lesson is narrower: a report needs to distinguish the opportunity to use a tool from what participants actually did with it.

For an agent, the relevant opportunity may be a specific eligible workflow rather than a patient visit. One encounter can create no suitable delegated task, while another can create several. Substituting visit counts for task eligibility would obscure that difference.

Define the denominator before choosing the headline

A proposed Heidi Work evaluation should begin with a workflow definition. For example, consider a fictional service offering patients a choice of replacement appointment slots after a clinician has already approved the change. Eligibility would depend on the permitted task, available information and local process, not merely whether the software can generate a message.

The evaluation would then distinguish eligible workflows, those authorised for delegation, those actually delegated and those reaching the agreed endpoint. Each transition answers a different question. Delegated work as a proportion of eligible work describes penetration into a suitable opportunity. Delegated work as a proportion of authorised work describes whether permission becomes use. Completed work as a proportion of delegated work describes execution within the selected workload.

These ratios should not be collapsed into a single adoption percentage. A high completion proportion could coexist with very limited use if clinicians delegate only a narrow subset of straightforward cases. Broad delegation could coexist with frequent failures or intensive rescue work. Neither pattern can be interpreted fairly without the other measures.

The endpoint must also be explicit. In the fictional appointment workflow, a message offering slots is not the same as a confirmed booking. A booking confirmation is not proof of attendance. The measure should match the work that was actually authorised.

Keep appropriate escalation visible without calling it completion

Some tasks should stop. A patient may raise a new concern, reject the available options or introduce information outside the agent's permitted scope. An appropriate escalation can demonstrate that a boundary worked even though the original workflow did not complete autonomously.

That is a reason to report escalations separately, not to quietly count every escalation as a successful completion. The report should distinguish completed work, appropriate handovers, unresolved work and failures. It should also describe the human effort required after handover.

Reopened tasks deserve attention. A workflow recorded as complete may later need correction because the intended outcome was not achieved. Reporting only the first successful status would miss that burden. A sensible evaluation would therefore define a follow-up period appropriate to the task and examine whether completion remained valid.

Likewise, generating more actions is not automatically beneficial. Repeated reminders, redundant messages or duplicate record entries can increase activity without improving care coordination. Appropriateness belongs alongside volume.

Read low and high engagement through the work underneath

There are four useful patterns to distinguish. Low interface use with verified completed work could indicate quiet, reliable delegation. Low interface use without delegated activity could indicate abandonment, lack of eligible work or a technical problem. High use while staff learn a new process could be a temporary implementation phase. High use because staff repeatedly rescue tasks could signal a product or workflow problem.

These are interpretations to test, not categories that can be assigned from screen time alone. Event records should be checked against the relevant destination systems and staff accounts. Interviews can establish whether a clinician is deliberately relying on the agent or has simply stopped thinking about it because another colleague now carries the work.

A particularly useful question is what happens when the original champion is absent. Does delegated work continue appropriately, does it stop because nobody else knows the process, or does it continue without adequate oversight? Similar usage totals could conceal very different organisational arrangements.

Renewal should follow sustained value, not visible busyness

For buyers, a credible renewal case would connect appropriate task completion with less unfinished work, manageable exceptions and acceptable patient experience. It would include the effort required to maintain the workflow, not just the time apparently removed from a clinician's screen.

For founders and investors, the useful commercial question is whether customers would miss the completed work if the service stopped. That requires evidence of repeatable value in a defined setting. It cannot be answered by moving engagement numbers from a conventional application into an agent dashboard unchanged.

Heidi Work's success might look quiet. But quiet is meaningful only when somebody can establish what was authorised, what happened and what still needs attention.

Frequently asked questions

Are fewer logins a good sign for an autonomous clinical agent?

They can be, when appropriate delegated work is continuing and reaching verified endpoints without unnecessary intervention. They can also indicate abandonment or lack of eligible work, so login counts alone are insufficient.

Which denominator should an adoption report use?

It should report clearly defined eligible, authorised and delegated workflows rather than presenting them as interchangeable populations. Completion and escalation should then be measured within the delegated workload, with the workflow type kept visible.

Should an appropriate escalation count as a completed task?

Not unless escalation was itself the authorised endpoint. It should normally be reported separately as a potentially appropriate boundary response, alongside the human work still required to resolve the original task.

Explore iatroX insights on evaluating clinical AI →

More from the Journal