The question "does AI tutoring work?" is unanswerable as posed, because it bundles four different tools and four different outcomes into one word. The tools: a generic chatbot, a structured tutor, a classical intelligent tutoring system, an adaptive learning platform and an AI feedback engine are different interventions with different mechanisms. The outcomes: performance while AI is available, immediate unassisted performance, delayed retention and transfer to new problems are different measurements that routinely point in different directions within the same study. This page is the canonical evidence hub for the whole pillar, replacing our earlier overview articles, and its organising rule is simple: never let an assisted-performance result masquerade as a learning result.
The mathematics experiment that defined the field's central caution
A field experiment published in PNAS ran nearly 1,000 Turkish high-school pupils through mathematics practice in three conditions. During AI-assisted practice, a generic GPT interface improved performance by 48% and a purpose-built, guarded GPT Tutor by 127%, enormous assisted gains. Then access was removed, and the finding that matters happened: the generic-GPT group performed 17% worse than controls on unassisted work, while the guarded Tutor group avoided that harm but did not outperform controls on the subsequent unassisted examination. The legitimate conclusion is therefore not "AI tutors improve learning by 127%", a misreading that now circulates widely; it is that tool design produces very large differences in performance while assistance is available, that unguarded answer access impaired subsequent independent performance in this study, and that guardrails prevented harm without, here, producing durable superiority. Assisted performance, immediate learning, delayed retention and transfer came apart in one experiment, which is exactly why they must be reported apart.
The physics trial that shows engineered pedagogy can work
A 2025 randomised crossover study in a university physics course found greater short-term learning in less time with a custom AI tutor than with an active-learning class, a genuinely strong comparator. The crucial detail is what the tutor was: not an ordinary chatbot but carefully sequenced content, established pedagogical principles, expert-developed solutions, structured prompts and substantial pre-deployment accuracy work. The study supports the proposition that well-engineered AI pedagogy can beat good teaching on short-term outcomes; it does not license the inference that handing students unrestricted ChatGPT reproduces the result, and most product marketing quietly makes exactly that inference. The working distinction this pillar uses everywhere: model capability, can the AI answer; product design, does the system induce the cognitive activity learning requires; and educational effectiveness, does that activity produce later retention or transfer. Most reviews collapse all three; the evidence separates them.
What appears to make an AI tutor educationally useful
Across the tutoring literature, the design features that recur in effective systems are unglamorous and consistent: requiring an attempt before help; progressive hints rather than immediate answers; diagnosis of the specific error; retrieval practice built into the loop; feedback tied to the learner's reasoning; and delayed retesting without assistance. The retrieval-and-spacing half has the firmest medical evidence of all: a 2026 medical-education meta-analysis of spaced repetition, 14 studies with 21,415 learners, found a substantial pooled benefit over standard studying, standardised mean difference 0.78, across heterogeneous implementations, which supports the mechanism strongly while validating no particular vendor's scheduler, including ours.
What the evidence does not yet show
Stated plainly because the hub's credibility depends on it: not universal superiority to human tutoring; not durable effects across subjects and populations, since most trials are short-term; and not product-specific superiority for any commercial platform, iatroX included, because pooled effects from varied interventions validate categories of mechanism, not implementations. The medical-education randomised evidence specifically is encouraging and immature, and has its own canonical analysis at /blog/what-20-randomised-trials-generative-ai-medical-education; the buyer's translation of all of it is a checklist, not a verdict: does the product require attempts, scaffold before telling, schedule retrieval, retest unassisted, and can the vendor show which outcome level its claims live at?
Frequently asked questions
So do AI tutors work or not?
Design-dependent and outcome-dependent: structured tutoring has produced real short-term learning gains against strong comparators, unguarded answer access has produced measurable harm to independent performance, and durable-retention evidence remains thin everywhere; any flatter answer is marketing.
Why does the 127% figure keep appearing?
Because it is true about the wrong thing: it measured performance with the tutor present, and its own study found no unassisted superiority afterwards; quoting it as a learning gain is the field's most common citation error.
How should iatroX's own design be judged against this?
Against the checklist, not our say-so: the platform implements attempt-first questioning, Socratic scaffolding, spaced retrieval and unassisted retesting because those are the evidenced mechanisms; whether our implementation improves your learning is a question our own forthcoming experiments, not this evidence hub, must answer.