An unseen question can measure your readiness exactly once; after that it measures only your memory of it. Most candidates spend this measurement asset without realising it is finite, drilling every available question early and arriving at the final fortnight with nothing cold left to tell them the truth. This is the assessment-design pillar of our framework series, and it teaches you to preserve unseen Q-bank questions as scarce assets: to reserve a holdout at the start, meter its consumption across the whole preparation, and only ever read your readiness from cold items. The practical outcome is that you keep an honest, uncontaminated signal of whether you are ready, all the way to the exam, on any format in any jurisdiction.
Why an unseen question is a depreciating asset
The core idea is borrowed from measurement science and is not controversial: you cannot validly assess an instrument on the same data you used to build it, which is why analysts hold out a test set they never train on. Applied to revision, your knowledge is the instrument and unseen questions are the holdout set — and every time you look at one, you spend it, because a seen question thereafter measures recognition rather than transferable knowledge. This depreciation is jurisdiction-independent and format-independent. A best-of-five item in MRCP(UK) Part 1, a next-step vignette in USMLE Step 2 CK, and a multiple-choice item in MCCQE Part I — which since April 2025 is multiple-choice throughout — all lose their measurement value the instant you have worked through them once. The scarcity is more acute the more elaborate the item: a written MCQ is one spent asset, but a Primum case simulation for USMLE Step 3 or a simulated consultation for MRCGP SCA is an even scarcer one, because there are far fewer of them and each is even more memorable once done. Treating these assets as if they were an inexhaustible stream of practice is the single most common reason a candidate's readiness signal quietly dies before the exam, and it is the asset-management version of the caution in the Q-bank percentage article.
The preserve-unseen method, stated precisely
The method is an asset-management discipline in five moves: Reserve, Ration, Replenish, Record, and Ring-fence the finish.
Reserve. At the very start of preparation, designate a holdout of unseen questions you will not drill — ideally a separate bank kept pristine, or a free measurement bank, or a locked subset of a large bank you commit never to open for practice. Decide its approximate size now, before familiarity makes every question feel spent.
Ration. Meter consumption. Decide how many unseen items you will spend per week as measurement blocks and hold to it, so you do not burn the reserve in an early burst of enthusiasm. Each unseen item is single-use for measurement; once spent, it moves to the learning pile.
Replenish. Where you can, top the reserve up — a vendor adding new questions, a second bank, or a bank whose selection draws widely across the blueprint so blocks rarely repeat — so the reserve does not run dry before the exam. Replenishment is what lets you measure weekly for months rather than a handful of times.
Record. Log each measurement block's first-attempt accuracy and its blueprint coverage as your readiness signal, and only this signal. These figures feed the blueprint coverage matrix and are the honest counterpart to your learning bank's recognition-inflated percentage.
Ring-fence the finish. Protect a block of genuinely cold questions for the final fortnight, when the most valuable reading of all — your readiness at exam pace, unassisted, days before the exam — is taken. This is the reserve within the reserve, and it is the first thing candidates raid and the last they should.
The unseen-question budget and the contamination checklist
Two tools make the method concrete. The first is a budget that treats unseen questions like a balance you draw down; copy it into your notes as the embeddable core every child article links back to.
| Weeks to exam | Reserve at start | Weekly ration | Running balance | Final-fortnight ring-fence | Status |
|---|---|---|---|---|---|
| 10 | e.g. 400 unseen | e.g. 30/week | 400 | 100 held back | On budget |
| 6 | — | 30/week | e.g. 280 | 100 held back | On budget |
| 2 | — | ring-fence only | 100 | 100 | Protected |
The second tool is a contamination checklist, because a reserve is only as large as the questions that are genuinely still unseen, and reserves shrink invisibly. An item is no longer a valid measurement asset if any of the following is true:
- You have drilled it, even once, in this or a previous cycle.
- You have met the same item, or a close clone, in another bank you use.
- A tutor, study partner or forum has shown you the item or its answer.
- An AI tutor leaked the answer or reasoning before you committed.
- The item is a repeat the bank re-served you without flagging it as seen.
Run the checklist before you count a block as measurement, because a reserve contaminated by any of these routes gives you the reassurance of an unseen reading without the validity of one.
Worked examples across three exam types
Written MCQ (MCCQE Part I, Canada). With ten weeks and a reserve of, say, four hundred unseen items across a free measurement bank, you ration thirty a week as a Monday timed block, log first-attempt accuracy and coverage, and ring-fence a hundred for the final fortnight. Because the bank draws widely across the content areas, each block also audits coverage, so the budget and the coverage matrix are filled from the same source. The learning happens in a different bank entirely; the reserve is never drilled. Ten weeks later you have a genuine readiness curve and a cold block left for the finish.
Structured response (RACGP KFP, Australia). Key Feature Problems — seventy multiple-selection cases in the Fellowship written stage — are scarcer than MCQs, and official specimen material is scarcer still, so the ration is tighter and the reserve smaller. Here the discipline is to resist the temptation to work every available KFP for practice, because the official and near-official cases are the highest-quality holdout you will ever have; spend them slowly, as measurement, and use bank-generated cases for the drilling. A candidate who burns the specimen papers in week two has destroyed their best measurement asset for a marginal amount of early practice.
Clinical simulation (USMLE Step 3 CCS and MRCGP SCA). Simulated cases are the scarcest assets of all and depreciate fastest, because a Primum case or an SCA rehearsal case is intensely memorable once done. The method here is almost entirely about ring-fencing: reserve a few cold cases for the final fortnight at all costs, because the reading you most need — can I run an unfamiliar case under time, unassisted, days before the exam — cannot be taken on a case you already remember. Drilling every simulation early feels like thorough preparation and leaves you, at the finish, with no honest way to measure the one skill the simulation exists to test.
Failure modes and where the method should not be applied
The method has failure modes at both extremes. Over-hoarding is a real one: a reserve you never open measures nothing, and a candidate so protective of unseen questions that they take only two measurement readings in a whole preparation has the opposite problem from the one this pillar addresses — the discipline is to meter consumption, not to freeze it. At the other extreme, treating a large bank as automatically a large reserve is a mistake, because contamination shrinks the true holdout far below the headline count, and an unaudited "unseen" block is often half-remembered. The reserve does not teach — do not try to learn from it, because the moment you review its explanations to study, you have converted a measurement asset into a learning one and it can no longer measure. And a reserve cannot fix a coverage problem: if your learning bank under-covers a domain, the reserve will faithfully reveal the weakness but not repair it, which is a job for targeted practice and the coverage matrix. Finally, the method assumes you have the discipline not to raid the ring-fence; a candidate who cannot resist should use a bank they have no incentive to drill — a free one — purely as the reserve, so the constraint becomes the safeguard.
The evidence and logic behind it
The method formalises two well-established ideas. The first is the train/test separation from measurement and statistics: a valid estimate of performance on new cases requires data the model has not seen, and an estimate taken on training data is optimistically biased — exactly the bias a rising percentage on a drilled bank exhibits. The second is the recognition-versus-recall distinction from memory research: recognising a previously-encountered item overstates the durable, transferable knowledge the exam actually tests, so only first-attempt performance on unseen items is a valid readiness signal. Neither idea is novel; the contribution of this pillar is to treat their practical consequence — that unseen questions are a scarce, non-renewable measurement resource — as something to be budgeted rather than assumed. Exam bodies themselves rely on unseen items precisely because seen items do not measure the construct; preserving your own holdout applies the same logic to your own preparation. Where any explanation you read while learning would change your behaviour clinically, verify it against a current, dated source before you rely on it, so the knowledge you carry into the measured block is correct.
An iatroX workflow that demonstrates the framework
iatroX is built to be the reserve, and its free UK-core banks make preserving a holdout close to costless — which is the honest reason to point to it here. The pattern: learn in whatever bank you prefer, and ring-fence iatroX as the measurement reserve you never drill, opening it only for timed, unseen, blueprint-spanning blocks. Because the core banks are free, replenishing and rationing the reserve does not add a second subscription's cost, and because iatroX draws questions widely across the blueprint, each block both measures accuracy and audits coverage for your matrix. When a block reveals a weak or blank domain, you take it back to your learning bank to drill and then measure again on a fresh block — the reserve and the learning bank staying cleanly separate. This is the two-Q-bank rule seen from the asset-management side: the second bank exists to be the preserved reserve. iatroX operates a competing bank, so we confine its role to the measurement job your learning bank cannot do for itself, and we make no proprietary-algorithm claims about how its blocks are drawn.
Implement it this week
Set your budget now, before another week of drilling erodes the reserve. Count the weeks to your exam, designate a reserve — ideally a free or separate bank you commit not to drill — and decide a weekly ration and a final-fortnight ring-fence. Run the contamination checklist over anything you are counting as unseen. Then take one measurement block this week: timed, cold, blueprint-mixed, first-attempt only, logged for accuracy and coverage and nothing else. Compare its first-attempt accuracy with your learning bank's headline percentage; the gap between them is the readiness illusion this pillar exists to dispel, and seeing it once is usually enough to make you protect the reserve for the rest of the preparation. From here on, the ring-fenced block is untouchable until the final fortnight, whatever the temptation.
A fully worked reserve, start to finish
Take a candidate twelve weeks out. In week one they designate a free adaptive bank as the reserve, estimate roughly five hundred usable unseen items, set a ration of thirty a week, and ring-fence a hundred and fifty for the final fortnight. Their first measurement block reads 58% first-attempt; their learning bank's headline says 71%. That thirteen-point gap reframes the whole preparation — the reserve is telling the truth the learning bank cannot. Weeks two to nine: they drill the learning bank hard and spend thirty reserve items a week on Monday measurement blocks, logging accuracy and coverage, never repeating a measured item. The reserve balance falls from five hundred toward the ring-fenced hundred and fifty; the measurement curve climbs, slowly and honestly, from 58% into the high sixties as real transfer improves, while the coverage matrix filled from the same blocks shows two stubborn blank domains that drilling then closes. Weeks ten to twelve: rationing stops and the ring-fenced block is opened for full timed simulation, cold, days before the exam — the single most informative reading of the whole run, and one that only exists because it was protected. At no point was the reserve drilled, so at no point did its signal degrade. The discipline delivered what no single drilled bank can: an honest readiness curve and a cold final reading.
The three ways candidates destroy the reserve
First, front-loading. Enthusiasm early in the preparation drives candidates to work through every available question fast, and by the time they think to measure, there is nothing unseen left — the reserve was spent as practice before it was ever recognised as an asset. Second, silent contamination. Using overlapping banks, reading forums with answers, or letting an AI tutor reveal reasoning before commitment all convert unseen items to seen ones without the candidate noticing, so a block that looks like measurement is really recognition; the contamination checklist exists to catch this. Third, raiding the ring-fence. In the anxious final fortnight, the protected block looks like exactly the practice material a nervous candidate wants, and spending it feels productive — but doing so trades away the most valuable reading in the whole preparation for a little late reassurance, and leaves the candidate walking into the exam with no idea how they perform cold. Protect against those three and the reserve does its job; relax any one and you are back to a single drilled bank with a comforting, meaningless percentage.
Frequently asked questions
Does this framework work for every medical exam? Yes, because it is about the measurement value of unseen items, which depreciates identically regardless of format or jurisdiction. The method applies to best-of-five MCQs, structured short-answer and key-feature papers, and clinical simulations across UK, US, Canadian, Australian and European exams. What changes per exam is the scarcity of the reserve — official structured-response and simulation material is far scarcer than MCQ material, so the ration is tighter and the ring-fence matters more — which is why each exam-specific guide states how scarce its unseen material is and links back here rather than reproducing the method.
How often should the framework be updated? The method is stable; what you update is the budget, continuously, and the contamination checklist, every time your resources change. Redraw the running balance weekly so you always know how much reserve remains and whether you are on track to reach the final fortnight with your ring-fence intact. The only structural reason to revisit the method is a change in how much unseen material you can access — a vendor adding questions, or a second bank joining the stack — which changes your reserve size and ration, not the five moves.
Which metrics are valid across different Q-banks? The valid, portable metric is first-attempt accuracy on unseen items from a protected reserve, read together with the blueprint coverage of those items. That figure measures the construct the exam tests and means the same thing in any bank. A bank's headline percentage and its accuracy on repeated or previously-seen questions are not valid readiness metrics and are not comparable across banks, because they are inflated by recognition; they belong to the learning bank, not the reserve, and should never be mistaken for a measurement.
How should AI-generated feedback be verified? Two things matter here. First, an AI tutor that reveals answers or reasoning before you commit is a contamination source — it silently spends your unseen items — so use it only after you have taken the measurement, never before. Second, verify any behaviour-changing clinical claim in AI feedback against a current, dated primary source such as NICE, CKS, SIGN or the SmPC/eMC before you act on it. For automatically scored answers, calibrate the number with the feedback-calibration protocol, whose own "preserve" step is this pillar in miniature.
How does iatroX implement the framework? iatroX is designed to be the preserved reserve: its free UK-core banks make holding a holdout close to costless, it draws questions widely across the blueprint so blocks measure and audit coverage together, and it is kept for timed, unseen, first-attempt blocks rather than drilling. iatroX operates a competing question bank, so we confine its role to the measurement job a learning bank cannot do for itself, keep the learning source and the reserve cleanly separate, and make no proprietary-algorithm claims about how the blocks are selected.
Why preserving unseen questions is the discipline that pays for itself
The reason to adopt this discipline is that the honest readiness signal it protects is the one thing money and effort cannot otherwise buy in the final weeks, when you most need it. A candidate who preserved a reserve can, days before the exam, take a cold reading and know where they stand; a candidate who spent everything early has, at exactly that moment, only a familiar bank's inflated percentage and a feeling. The willingness to leave good questions unused for drilling feels wasteful and is in fact the entire source of the method's value — unused-for-learning is precisely what makes a question valid for measuring, and a reserve is only ever spent well when it is spent slowly. This is why the method is deliberately exam-agnostic: the specific banks and formats will change, candidates will sit exams that do not yet exist, and the fact that a question measures readiness exactly once will not. Reduce the pillar to one instruction and it is this: treat unseen questions as a non-renewable measurement asset, budget them like one, and never spend the final-fortnight reserve early — because the cold reading you protect is the only one that will still tell you the truth when it counts. The exam-specific guides in this series state how scarce each exam's unseen material is and how to ration it, and you can compare banks on the size and quality of the reserve they offer at the comparison hub; build the budget once here and every future exam becomes a matter of sizing a new reserve, which is the portability a good framework should give you.
Editorial notes and references
Written by Dr Kolawole Tytler, NHS GP and founder of iatroX. Last checked 19 July 2026. This is a cross-exam framework article; exam-specific guides in this series apply the method with named banks and state how scarce each exam's unseen material is, and any vendor figures they cite are dated and labelled vendor-reported. Disclosure: iatroX operates question banks — including free UK-core banks positioned here as the preserved measurement reserve — and a citation-first clinical AI; it is not a consultation or case simulator, its role is confined to the measurement job a learning bank cannot do for itself, and we make no proprietary-algorithm claims. Corrections via the feedback route on iatrox.com. References: train/test separation and validation principles from measurement science; recognition-versus-recall and the testing effect from memory research; official exam-body formats for MRCP(UK) Part 1, USMLE Step 2 CK and Step 3 CCS, MCCQE Part I, RACGP KFP and MRCGP SCA as cited in the relevant child articles; related reading: why your Q-bank percentage is not your exam score and the sibling pillars on the error taxonomy and transfer practice.
Set your unseen-question budget, then run one cold measurement block in iatroX →