A high medical examination score does not prove that an AI agent can conduct a consultation. A written question supplies information selected by somebody else; an interactive system must decide what to ask, which tools to use and when the available evidence is sufficient. AgentClinic makes that difference an explicit evaluation problem rather than treating a correct final answer as the whole task.
The AgentClinic paper, published in npj Digital Medicine on 27 April 2026, evaluates interactive, tool-using clinical agents in a simulated environment. It provides evidence about that environment, not prospective proof that a model can safely manage patients in routine practice.
An examination stem has already solved part of the problem
A well-constructed written question can be difficult while still doing substantial organisational work for the candidate. It provides a chronology, chooses findings, includes relevant investigation results and defines the decision being requested. The learner must interpret the supplied information, but does not necessarily decide how to obtain it.
An interactive encounter adds a different burden. Information may be incomplete, expressed indirectly or available only after an appropriate question. The first explanation may need revising. A tool might provide a useful result only if the agent selects it for the right reason.
This does not make written examinations worthless. It means their evidence should remain attached to the competence they assess. Recognition, recall, application and information gathering overlap, but they are not identical activities. A strong result on one task cannot silently become evidence for all the others.
What AgentClinic changes about the test
The April 2026 research report describes an environment involving simulated patient interaction, clinical tools and incomplete information. Its framework separates roles involved in the encounter and assesses how an agent proceeds through a case rather than giving it only a finished stem.
That design allows researchers to examine choices made along the way. Did the system ask for information that could distinguish competing explanations? Did it use an available tool appropriately? Did it continue gathering information after the task was adequately resolved, or stop before a material uncertainty had been addressed?
These are questions about process as well as outcome. An agent may arrive at the intended diagnosis through a fragile route. Another may ask sensible questions but fail to integrate the answers. Treating both solely as right or wrong final labels would hide different improvement needs.
The sequence can change what becomes visible
Consider an original fictional teaching case involving a person who reports intermittent tiredness. A static stem could supply the duration, associated symptoms, relevant history and test results. The candidate's task would then be to interpret that assembled picture.
In an interactive version, the learner first has to clarify what tiredness means to the person. Is the central difficulty sleepiness, reduced exercise tolerance, weakness or loss of motivation? That distinction can change the next question and the information that deserves attention.
Suppose an agent settles too early on one explanation and asks only questions that fit it. Later answers may create a coherent story without testing the alternative. The final response could sound clinically fluent while the investigation remains incomplete.
The educational question is not to list every possible cause. It is to identify the next question most likely to change the interpretation. A useful assessment could ask the learner to explain that choice before revealing the answer, then revise the plan when new information arrives. This is a proposed teaching exercise, not a reported AgentClinic case or result.
Tool use is more than access to a tool
A model that can call a search function or request an investigation has acquired a capability, not a guarantee of good judgement. The evaluation should examine whether the tool was needed, whether the request matched the uncertainty and whether its output altered the reasoning appropriately.
An agent could use many tools and still fail to answer the important question. It could also appear efficient by making few requests while overlooking information that should have been obtained. Neither tool-call volume nor apparent speed is a reliable substitute for appropriateness.
For an assessment designer, a useful method is to identify the decision each tool result is meant to support. Before obtaining the result, ask what different outcomes would change. Afterwards, examine whether the system actually responds to those differences. That tests purposeful information gathering rather than decorative investigation.
What simulation captures well, and what it cannot establish alone
A simulated environment can present comparable challenges repeatedly and preserve the sequence for inspection. It can expose an agent to deliberately incomplete histories or unusual combinations without putting a real patient through an experimental workflow. Those are useful methodological advantages.
However, the simulated patient is also a model of behaviour. Its willingness to disclose information, consistency, language and response to unclear questions may not match every real person. Tools and records are likewise representations of a clinical environment, not the entire environment itself.
The April 2026 AgentClinic study should therefore be read as a benchmark of specified interactive tasks. A deployment evaluation still needs to address real communication, accessibility, records, human supervision, local pathways and outcomes. These are additional questions, not reasons to dismiss simulation.
A particularly important boundary concerns physical practice. Describing an examination manoeuvre is not performing it. Correctly naming a possible sign is not demonstrating that it can be elicited or recognised in a real patient. The same distinction applies when evaluating machines and when designing doctors' training.
A more useful evidence request for buyers
A buyer should ask a supplier to identify which part of the consultation its evidence covers. A static benchmark might support a claim about answering prepared questions. An interactive benchmark might support a narrower claim about information gathering within a simulation. Neither should be presented as deployment clearance.
The next question concerns the tested system. Was it the same model version, instructions, tools and context that the organisation will use? A benchmark cannot automatically travel with a brand name when the surrounding workflow changes.
Finally, ask for error analysis. A result is more useful when it shows whether failures arose from missed questions, misread answers, unsuitable tool choices, premature closure or incorrect synthesis. Different mechanisms call for different controls and different types of human review.
What medical learners can take from the distinction
For learners, the implication is to combine forms of practice rather than abandon question banks. Written questions can help identify knowledge gaps. Guided follow-up can reveal why an attractive answer was wrong. Interactive cases can require the learner to decide what information is needed before committing to a plan.
As described in iatroX's September 2026 product specification, its question banks use adaptive sequencing and spaced repetition, while the Socratic Tutor starts from an attempted question and explores the learner's reasoning. Its examination-specific simulations provide voice or text practice with feedback linked to the interaction. These are complementary educational designs, not proof that every learner outcome improves.
A proposed sequence is to attempt a question independently, explain the discriminating feature, practise a different case without the complete history supplied and then review the choice of questions. A supervisor can examine whether the learner understood the process rather than merely recognising the final answer.
The endpoint is not an impressive score displayed in isolation. It is a clearer account of what was known, what remained uncertain, why the next step was chosen and what would justify changing direction. That is the useful bridge between AgentClinic's research question and everyday medical education.
Frequently asked questions
Is AgentClinic a clinical trial of an autonomous doctor?
No. The April 2026 paper evaluates agents in a simulated interactive environment, not a prospective care service treating real patients.
Why is a medical examination score insufficient for assessing consultation ability?
A prepared examination stem supplies selected information, while a consultation also requires the system to obtain, sequence and interpret information that may initially be missing.
Should learners replace question banks with simulations?
No. Question practice, guided reasoning, interactive cases and supervised clinical work address overlapping but different learning needs, and digital simulation does not replace bedside or procedural practice.