This review is for family medicine residents weighing SampQs' AI-graded SAMP practice for the CCFP examination. It leads with two findings that shape everything else: SampQs covers the SAMP written component only — there is no SOO product at all — and its core feature, instant AI grading of free-text answers, is built around exactly the response format the CFPC is now transitioning away from. Used with discipline it is a useful reasoning aid; relied on uncritically it can grade you confidently against the wrong format.
What SampQs offers for CCFP right now
Figures below are vendor-reported from the SampQs site and last checked on 19 July 2026. Confirm them on the product page.
| Item | What the site states (vendor-reported, 19 July 2026) |
|---|---|
| Positioning | "AI-Powered CCFP SAMPs Exam Preparation" |
| Content | 2,400+ SAMP questions; 598+ clinical cases; 105 priority topics |
| AI feature | "Instant expert grading" — write a free-text answer and get immediate feedback on marks earned and missed |
| Components covered | SAMPs only; no SOO product |
| Access | 30 free practice cases advertised, no credit card required |
| Pricing | Not published on the page — verify the current price on the product page |
The AI grader is the whole proposition, and it is a genuinely useful idea: free-text SAMP practice normally needs a human to mark it, and instant feedback lowers the friction of doing more cases. The audit question is whether that grading is well-grounded, whether it trains the right reasoning, and whether it is fidelity-matched to the exam you will actually sit.
The exam you are actually preparing for
The CCFP certification has two components: the SAMPs (computer-based at Prometric, about four hours of testing, case stems each with usually three to five questions) and the SOOs (five 15-minute virtual simulated office orals scored on a patient-centred framework). SampQs addresses the first and not the second. Critically, from April 2026 the CFPC is transitioning the SAMP away from written free-text toward multiple-choice and short-menu formats — publicly signalled as up to around a quarter of cases in Fall 2026 and all cases by Spring 2027, though exact counts and scoring are not published, so verify the current format and timeline for your diet on cfpc.ca. That transition is directly relevant to a product whose distinctive value is grading free-text answers.
Testing methodology: a fixed, publishable rubric
To review an AI grader fairly, run a fixed set of representative items through it and score its behaviour on a rubric you publish, so the assessment is repeatable rather than anecdotal. Use six item types — straight recall, diagnosis, next investigation, management, ethics/professionalism and one deliberately ambiguous case — and rate the tutor on each for: correctness of the model answer, quality of the mark breakdown, source grounding, handling of uncertainty, and jurisdictional fidelity. Do this without reproducing any copyrighted question or answer text; describe the item type and the behaviour, not the content.
Grounding audit: where do the marks come from?
The single most important question for any AI grader is provenance. When SampQs awards or withholds a mark, what is that judgement based on — a fixed, human-authored marking key for that case, an external guideline, or uncited model output generated on the fly? These have very different reliability. A fixed key written by clinicians is auditable and stable; live model output is neither. To inspect provenance without reproducing content, probe it: ask the grader to state the source and date of the standard it marked against, re-submit a near-identical answer and see whether the marks are stable, and submit a deliberately wrong-but-plausible answer to see whether it is scored against a rubric or merely pattern-matched. Instability across identical submissions is a strong sign of ungrounded, generative marking. Record what you find; do not assume a clinician-authored key exists just because the feedback reads fluently.
Reasoning behaviour: tutor or answer key?
A good tutor develops your thinking; a poor one hands you the answer. Assess whether SampQs' feedback asks or implies useful diagnostic questions, or whether it simply reveals the model answer immediately; whether it handles genuine uncertainty (some SAMP key features are legitimately contested) or forces a single "correct" line; and whether it corrects a false premise in your answer or grades around it. Submit an answer built on a wrong assumption and watch what happens: a strong grader flags the flawed premise, a weak one deducts marks for the downstream conclusions without ever naming the root error. The behaviour you want is one that makes you reason better next time, not one that just tells you the score.
Exam fidelity: jurisdiction, terminology, timing and weighting
Fidelity is where a Canadian family-medicine AI tool most often slips. Test four things. Jurisdiction: does the feedback reflect Canadian screening, first-line therapy and preventive-care norms, or does it drift toward American or generic guidance? Terminology: does it use CFPC framing — priority topics, key features, selectivity — or generic exam language? Timing: does it help you practise the selectivity the real SAMP rewards under time pressure, or reward exhaustive lists that would cost you time in the exam? Blueprint weighting: does its case distribution track the CFPC priority topics, or over-index on a few well-covered areas? And overarching all of these: format fidelity. A grader tuned to reward well-structured free-text prose is training a mode the SAMP is phasing out; the clinical reasoning transfers to MCQ and short-menu, but the answer-construction skill it grades may not. Weigh that against your diet.
Failure modes to watch for
- Hallucinated citations: confident references to guidelines that do not say what is claimed, or do not exist. Always verify a cited source.
- Overconfident wording: definitive language on points that are genuinely contested in family medicine.
- Outdated guidance: marking against a superseded screening interval or first-line agent.
- Answer leakage: revealing the model answer before you have committed yours, destroying the retrieval value of the case.
- Plausible-but-unexamined elaboration: fluent detail that sounds authoritative but was never checked against the CFPC key features the case is meant to test.
Safe-use protocol: answer first, interrogate second, verify third
A three-step routine keeps the tool useful and honest.
- Answer first. Write and commit your full free-text answer before you submit it for grading. No peeking; the retrieval only counts if you decide before the AI speaks.
- Interrogate second. After grading, drive the tutor with structured prompts rather than accepting the score: "Which key feature did I miss, and why does it matter?", "What false premise, if any, is in my answer?", "Name the Canadian guideline and date behind this mark."
- Verify third. For any guidance-sensitive point, check the claim against a current Canadian source yourself — the Canadian Task Force on Preventive Health Care, Diabetes Canada, Hypertension Canada or the relevant national guideline — and log discrepancies. Run this as a weekly sample so drift is caught early.
Worked example: a seven-day pattern for a busy trainee
A resident revising around clinical work, ten weeks out. SampQs does one job — free-text SAMP reasoning with instant grading — and iatroX does one job: unseen transfer practice and a fresh timed baseline, with no proprietary-algorithm claim on either side.
- Monday–Tuesday: two SampQs cases per day on priority topics you have flagged as weak; commit answers before grading; interrogate each with the key-feature and jurisdiction prompts.
- Wednesday: verify the week's guidance-sensitive claims against current Canadian sources; log any the grader got wrong.
- Thursday: transfer day — a fresh, unseen iatroX item on each principle before revisiting the SampQs case; note which survive transfer.
- Friday: SOO role-play with a partner — the component SampQs does not cover at all.
- Saturday: a timed SampQs set on unseen cases, then a short unseen iatroX block to confirm gains are not tool-specific; if your diet is MCQ/short-menu, spend part of this session on format-current practice.
- Sunday: rest or light review of the misconception list only.
Decision checklist: continue, supplement, switch or stop
- Continue if the grader passes your grounding checks (stable, sourced marks), its jurisdiction fidelity holds up, and your diet still uses enough free-text SAMP to make free-text grading relevant.
- Supplement with a partner for SOO role-play (mandatory — SampQs has no SOO) and with format-current MCQ/short-menu practice as your diet shifts.
- Switch or de-prioritise if identical submissions produce unstable marks, jurisdiction checks repeatedly fail, or your diet has moved fully to MCQ/short-menu and free-text grading no longer matches the exam.
- Stop relying on it for readiness once coverage, unseen performance and recency are met; the grader is a build-phase tool, not a mock.
Bottom line
SampQs solves a real problem — instant feedback on free-text SAMP answers — and for the current written format it can be a useful reasoning aid, provided you answer first, interrogate the grading, and verify guidance-sensitive claims against Canadian sources. But be clear-eyed about two structural limits: it does not touch the SOO, so you still need role-play, and its signature feature is tuned to a free-text format the CFPC is phasing out from April 2026. Confirm your diet's format on cfpc.ca, treat the AI as a tool to be audited rather than trusted, and keep an unseen-measurement layer so your score is never the grader's word alone.
Frequently asked questions
Is SampQs enough for CCFP on its own? No. It addresses only the SAMP component and has no SOO product, and its free-text grading is matched to a format the CFPC is transitioning away from — so even for the written paper it should be paired with format-current practice, and the SOO must be prepared separately through role-play or a course. As a SAMP reasoning aid it can be a useful part of a stack, not the whole of it.
Which CCFP component does SampQs not reproduce well? The SOO entirely — there is no oral, patient-centred simulation product — and, increasingly, the SAMP's answer format itself, because instant grading of free-text prose is tuned to a mode the exam is moving toward MCQ and short-menu. Its clinical reasoning value transfers; its answer-construction grading may not, depending on your diet.
How should I verify SampQs AI answers for CCFP? Commit your own answer first, then ask the grader to name the Canadian guideline and date behind each mark, re-submit near-identical answers to check the marks are stable rather than generative, and independently verify any guidance-sensitive claim against a current Canadian source, logging discrepancies weekly. Unstable marks on identical answers are a sign of ungrounded, uncited model output.
When should I stop using SampQs and move to mixed mocks? Treat SampQs as a build-phase reasoning tool and move to timed, mixed practice once your priority-topic coverage and recency checks are met; if your diet has shifted to MCQ/short-menu, your mocks should use that format rather than free-text, so confirm the format on cfpc.ca first. Begin SOO role-play in parallel throughout, since SampQs cannot help there.
How should I combine SampQs with iatroX without duplicating practice? Assign each one job: SampQs for free-text SAMP reasoning with grading, iatroX for unseen transfer practice and a fresh timed baseline, so you never re-answer a case you have already seen graded. That preserves an independent measurement layer — the two-bank principle — and neither tool claims a proprietary algorithm; iatroX is not an SOO simulator either, so keep role-play separate.
Editorial notes and references
Written by Dr Kolawole Tytler, NHS GP and founder of iatroX. Last checked 19 July 2026. Vendor figures (question and case counts, priority-topic count, the AI grading claim, the free-trial offer) are vendor-reported from the SampQs site on that date and change without notice — pricing was not published, so verify the current price on the product page, and verify the CCFP format and 2026 transition timeline for your diet on cfpc.ca. Disclosure: iatroX operates a competing CCFP question bank and a Socratic Tutor, so this article confines iatroX's role to unseen transfer practice and timed measurement — jobs SampQs does not claim — and makes no proprietary-algorithm claims and does not position iatroX as an SOO simulator. Corrections are welcome via the feedback route on iatrox.com.
References: College of Family Physicians of Canada, Certification Examination in Family Medicine, SAMP and preparing-for-the-examination pages, and the Priority Topics and Key Features (cfpc.ca); SampQs product page (sampqs.com). Related iatroX reading: the CCFP bank landing page (iatrox.com/canada/exam/ca-ccfp), "Your Q-Bank Percentage Is Not Your Exam Score", the guide to auditing an AI medical-exam tutor (grounding, answer leakage, hallucinations and retention), and the guide to calibrating AI-graded feedback before you trust the score.