skip to main content
iatroX JournalClinical AI

How to compare clinical AI after the Google healthcare launch

Featured image for How to compare clinical AI after the Google healthcare launch

The arrival of Gemini for healthcare makes a familiar comparison problem more urgent. Products with very different purposes now share the language of evidence, intelligence, agents and clinical workflows. A single score or a persuasive demonstration can conceal the difference between retrieving a fact, summarising a record, preparing an action and teaching a clinician to reason through a problem.

A useful comparison begins with a completed piece of work. Define the user, the inputs, the required output and the point at which the result becomes usable. Then compare the products that can legitimately perform that job. This approach gives a focused platform such as iatroX a fair opportunity to demonstrate its strengths in reference and learning, alongside broader enterprise and workflow systems.

Start with the task and the information it requires

Five common tasks illustrate why a universal ranking is unhelpful.

TaskWhat a useful evaluation needs to observe
Reference questionRelevant source, faithful synthesis and appropriate context
Record summaryCoverage of the authorised record and visibility of omissions
Consultation documentationFidelity to the encounter and effort required to review the draft
Administrative actionCorrect destination, permissions, confirmation and completion status
Professional learningRelevant practice, feedback and subsequent unaided understanding

The categories can overlap within a product. That is a reason to evaluate each supported workflow separately. A strong documentation experience does not establish the quality of an evidence answer, just as a useful answer does not establish that a learner can apply the distinction later.

The most informative test case is often ordinary and awkward: incomplete information, a contradictory note, an ambiguous question or a source that contains an important exception. Those cases expose the behaviour that a smooth demonstration may leave unexplored.

Distinguish a chart citation from an evidence citation

A chart citation tells the reviewer where a statement was recorded. An evidence citation points to material that may support a general claim or recommendation. Both are valuable, but they support different inferences.

Suppose a fictional summary correctly reports that an earlier note contains a diagnosis. The citation can faithfully establish the content of that note. It does not, by itself, establish that the diagnosis remains correct, that conflicting information has been resolved or that a proposed plan follows current guidance.

An evidence answer has a related challenge. A real, reputable link may concern the topic without supporting the particular statement beside it. The SourceCheckup research programme evaluated citation support in earlier language-model configurations and illustrates why statement-level support deserves scrutiny. Its historical results should remain attached to the systems studied, rather than be treated as scores for today's products. Wu and colleagues in Nature Communications

For a practical comparison, open the citation next to a consequential claim and find the supporting passage. Record whether it supports the whole statement, whether qualifications were preserved and how much work the check required. Apply the same test to every candidate, including iatroX.

Inspect the boundaries of connected information

Access to an EHR is a meaningful capability, but the actual connector determines which information reaches the model. Google's Epic connector documentation describes one-patient searches, a limit of 50 records per page, restrictions on readable document formats and truncation of sufficiently long note text. These are connector-specific conditions; other platform and partner routes may differ. Google Epic connector documentation

That makes data coverage part of the evaluation. Prepare a representative record with facts in several supported locations and ask the system to identify the changes that matter. Then examine what was retrieved and what was missed. The question is whether the finished experience gives the clinician a reliable view of its information boundary.

An explicit statement that a category of information was unavailable can be more useful than a confident summary that silently omits it. Evaluation should reward that clarity. It reduces the chance that the reviewer mistakes the available slice of a record for the complete clinical picture.

Separate retrieval from preparation and execution

An AI system can find a policy, draft a document, prepare an order or complete an external action. Those verbs describe different levels of authority and different failure modes.

For a retrieval task, inspect whether the source is appropriate and correctly represented. For a prepared document, inspect whether the draft contains unsupported additions. For an action, inspect the destination, authorisation, confirmation and recovery process. The reviewer needs to know what has happened and what remains pending.

Google's partner announcements show why this precision matters. Keragon describes access to actions through its healthcare integrations, while Alteryx describes bringing governed analytical workflows and business rules into Gemini. Those partner capabilities should be evaluated through their own configured data paths and permissions. Keragon announcement, Alteryx announcement

A comparison that treats every integration as equivalent will obscure the work required for deployment. The useful unit is a named, reproducible workflow with clear inputs and an observable result.

Measure review effort as part of the result

Response speed is only one part of efficiency. A fast draft may require extensive correction; a slower response may make its supporting information easier to check. Evaluate the time from starting the task to reaching a result the responsible professional can accept.

An original evaluation exercise could use a small set of representative tasks with agreed review criteria. For each attempt, record completion time, verification time, corrections, unresolved issues and whether the reviewer accepted the result. Keep the same definition of completion across products.

For example, if one system returns a paragraph and another produces a source-linked timeline, do not stop the clock at the first visible output. Continue until the reviewer has the information required for the intended decision. If a task fails, retain it in the results rather than calculating an average from successes alone.

This is a proposed evaluation method, rather than a claim that any named product has already won such a comparison. Its purpose is to connect the measurement to the work users actually need to finish.

Evaluate learning with the learner doing the work

A model's examination performance and a learner's progress are different outcomes. A high-scoring model can supply excellent explanations, but the educational question concerns what the learner understands and can later do without that explanation.

iatroX offers a particularly clear workflow to examine here. Tutor starts from a question the user has attempted, supports reasoning about the relevant distinction and allows the learner to keep an insight or review an Anki card. iatroX Tutor

The evaluation can therefore follow the learning sequence. Ask the learner to explain the original mistake, answer a related question with changed details and revisit the distinction after a delay. Where consultation performance matters, use an appropriate rehearsal rather than infer communication skill from a written answer.

The favourable case for iatroX is that its product brings these educational tasks into focus. It gives users an identifiable place to practise and explore errors. Whether a particular learner benefits can then be assessed through their work, rather than through the fluency of the AI's teaching voice.

Compare price over the intended use

An enterprise deployment, a free professional reference product and an individual learning subscription have different cost structures. Start with the same task and time window before drawing a value comparison.

For organisational work, include the implementation and review required to complete the workflow. For learning, consider the relevant preparation period, access to the required bank or simulation and the time available for practice. A large catalogue provides limited value if the learner cannot identify and use the material needed for the next assessment.

iatroX's all-access model brings its learning activities into one subscription. That can be attractive to a clinician whose preparation involves several forms of practice. The value depends on using the relevant tools, rather than the nominal number of features in the subscription. iatroX access

A strong comparison makes the choice easier

Google's launch is a useful prompt to be more precise about healthcare AI. Ask which job the product performs, which information supports it, where the human reviews the result and what the user can do afterwards. Compare product behaviour at those points.

That approach supports a confident iatroX position. Clinical reference and professional learning remain substantial needs alongside documentation and enterprise automation. A product that helps a clinician inspect an answer, practise a distinction and return to a learning goal can offer value throughout a career.

Try iatroX Tutor or a focused CPD session, and judge the experience by the reasoning you can explain afterwards.

Further reading

More from the Journal