Inter-rater reliability — MRCPsych Paper A MCQ

Instant feedback + full explanation. One question, done properly.

EasyEpidemiology and research methodsInter-rater reliabilityMRCPsych Paper A

Two psychiatrists independently assess the same interview recordings. For each recording, they classify catatonia as either present or absent. Which statistic is most appropriate for quantifying agreement between the psychiatrists?

Educational content. Not a substitute for clinical judgement or local policy.

Reveal the answer and explanation

Correct answer: E — Cohen kappa

Cohen kappa is the best answer. It quantifies agreement between two raters making categorical assessments and adjusts the observed agreement for that expected by chance. Here, the two psychiatrists independently classify the same recordings using a binary outcome: catatonia present or absent. Cronbach alpha assesses the internal consistency of multiple items within a scale rather than agreement between raters. A hazard ratio compares event rates over time, while an odds ratio measures the association between an exposure and an outcome. A standardised mortality ratio compares observed with expected deaths in a population. If the psychiatrists were instead assigning a continuous total score, an intraclass correlation coefficient would usually be considered, which is why the binary nature of the final stem is made explicit.

Reference: McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb). 2012;22(3):276–282. https://pubmed.ncbi.nlm.nih.gov/23092060/