Certainty and safe consequence responses provide additional information from multiple choice question assessments

M.J. Tweed, S. Stein, T.J. Wilkinson, G. Purdie, J. Smith

Clinicians making decisions require the ability to self-monitor and evaluate their certainty of being correct while being mindful of the potential consequences of alternative actions. For clinical students, this ability could be inferred from their responses to multiple-choice questions (MCQ) by recording their certainty in correctness and avoidance of options that are potentially unsafe. Response certainty was assessed for fifth year medical students (n = 330) during a summative MCQ examination by having students indicate their certainty in each response they gave on the exam. Incorrect responses were classified as to their inherent level of safeness by an expert panel (response consequence). Analyses compared response certainty, response consequence across student performance groupings. As students' certainty in responses increased, the odds they answered correctly increased and the odds of giving unsafe answers decreased. However, from some ability groups the odds of an incorrect response being unsafe increased with high certainty. Certainty in, and safeness of, MCQ responses can provide additional information to the traditional measure of a number correct. In this sample, even students below standard demonstrated appropriate certainty. However, apart from those scoring lowest, student's incorrect responses were more likely to be unsafe when they expressed high certainty. These findings suggest that measures of certainty and consequence are somewhat independent of the number of correct responses to MCQs and could provide useful extra information particularly for those close to the pass-fail threshold.