How 19,000+ UK Clinicians Actually Used an AI Clinical Reference (iatroX 2025 Formative Evaluation)
A mixed-methods study of the iatroX platform has provided critical, early real-world signals on the adoption, usability, and perceived value of a UK-gated, provenance-first AI clinical reference. The findings, drawn from retrospective usage data and an anonymous 1,223-respondent user survey, suggest that clinicians will quickly adopt artificial intelligence tools that are fast, free at the point of use, and clearly grounded in trusted UK guidance (NICE, CKS, SIGN, BNF).
This summary is based on the preprint, "Adoption, usability and perceived clinical value of a UK AI clinical reference platform: a mixed-methods formative evaluation of real-world usage and a 1,223-respondent user survey" (arXiv:2509.21188, April–July 2025 data).
1) Why We Ran This Study
UK Clinicians face simultaneous information overload from rapidly changing national guidelines and complex local protocols. Traditional clinical decision support (CDS) tools are often too slow or require too many clicks at the point of care.
Our Hypothesis was that a UK-centred, Retrieval-Augmented Generation (RAG)-based platform that always cites its sources can successfully reduce cognitive load and be adopted quickly by a diverse clinical workforce.
The Aim of this formative study was to describe the iatroX design and report early real-world signals on its adoption, usability, and perceived clinical value among UK clinicians.
2) What iatroX Actually Is (Technical Overview in Plain English)
iatroX is built as a UK-gated RAG pipeline, ensuring all generated answers are traceable to authoritative sources:
- Retrieve: it searches from a curated knowledge base of NICE, CKS, SIGN, and BNF content.
- Generate: it extends its search scope if necessary and applies a safety threshold to the retrieval.
- Refusal: if confidence is low or the query is outside the scope of UK guidance, it can refuse to generate a definitive answer, preferring safe abstention over confident hallucination.
The platform uses proprietary algorithmic search and vector retrieval over approximately 500-token chunks. iatroX is registered with the MHRA as a Class I medical device (Ref: 2025042201417535) and is available via web, iOS, and Android frontends.
3) Adoption: What Happened in 16 Weeks
The study analysed usage data across a 16-week period (8 April – 31 July 2025), revealing rapid and engaged adoption signals:
- Total Users and Queries: there were 19,269 unique web users who submitted approximately 40,000 clinical queries across all platforms.
- Engagement: the platform logged 202,660 engagement events, equating to roughly 10.5 events per active user, suggesting repeat, not one-off, use.
- Mobile Use: the iOS app saw 1,960 downloads, with the Android Daily Active User (DAU) count peaking at over 750.
The Key Takeaway for readers is that clinicians will use AI Clinical Decision Support (CDS) if it is fast, UK-specific, and offered free at the point of use.
4) Usability & Perceived Value: Headline Survey Signals
The in-product intercept survey, which used a single-item randomised prompt design (N varied per item), returned highly positive results regarding the platform’s perceived value:
| Survey Item (95% Wilson CI) | Positive Response % |
|---|---|
| Usefulness | 86.2% |
| Would Use Again | 93.3% |
| Willing to Recommend | 88.4% |
| Perceived Accuracy | 75.0% |
| Perceived Reliability | 79.4% |
| Time Saved | 60.9% |
(Note: these results are from the anonymous intercept survey, as detailed in the arXiv preprint.)
5) What Users Actually Liked (Qualitative Themes)
A rapid thematic analysis of open-ended and unsolicited feedback highlighted four main areas of user satisfaction:
- Speed: users frequently cited that answers were delivered in approximately 12 seconds thanks to the safety-aware RAG flow, significantly faster than traditional guidance search.
- Guideline-Grounded: outputs were highly valued for being clearly tied to specific UK sources (NICE/CKS/SIGN/BNF) rather than generic web searches.
- UK Specificity: unlike general-purpose large language models (LLMs), iatroX consistently provided UK-centric advice, which is critical for prescribing and management.
- Free & Mobile: low-friction entry and availability on mobile were cited as essential for GPs, trainees, and PAs who are frequently moving between clinics and wards.
6) Why This Matters for NHS/DHSC Audiences
This study provides early, robust evidence that is directly relevant to policymakers and digital leads:
- Measurable Uptake: it shows real, measurable uptake of a provenance-first AI CDS tool within UK primary and community care settings.
- Safety Assurance: it demonstrates that RAG with refusal-to-answer is acceptable to users; they clearly prefer safe abstention to confident hallucinations from the tool.
- Regulatory Alignment: the documented pipeline, sources, and usage metrics provide essential evidence for subsequent DTAC (Digital Technology Assessment Criteria), NICE EVA, and MHRA AI Airlock conversations.
7) What’s Next: Future Research
While highly positive, the study cautioned readers about small per-item survey sizes and potential early-adopter bias. Future work is already planned to build on these signals:
- Objective Accuracy Audit: formal, objective audits are planned to compare iatroX outputs against gold-standard clinical scenarios.
- Workflow Impact: randomised controlled trials (RCTs) will be conducted to objectively measure the impact on time-to-answer and diagnostic workflow compared to using traditional search methods.
- Comparisons: further research will compare the performance of iatroX with other AI clinical references like OpenEvidence and Dyna AI.
The full analysis is available for download via the arXiv preprint, offering a transparent look at the future of NHS-ready AI.
8) Calls to Action
- Clinicians: try iatroX on a real clinical query on your next shift and tell us if the source is correct.
- Educators/ICBs: use these early metrics as evidence that UK staff will engage with AI if the output is demonstrably citation-first.
- Researchers: collaborate with us on the planned accuracy and workflow RCTs to strengthen the evidence base for clinical AI adoption in the NHS.