↓ Skip to main content

Impact of question order on prioritisation of outcomes in the development of a core outcome set: a randomised controlled trial

Overview of attention for article published in Trials, January 2018
Altmetric Badge

Mentioned by

twitter
32 X users

Citations

dimensions_citation
25 Dimensions

Readers on

mendeley
80 Mendeley
You are seeing a free-to-access but limited selection of the activity Altmetric has collected about this research output. Click here to find out more.
Title
Impact of question order on prioritisation of outcomes in the development of a core outcome set: a randomised controlled trial
Published in
Trials, January 2018
DOI 10.1186/s13063-017-2405-6
Pubmed ID
Authors

Sara T. Brookes, Katy A. Chalmers, Kerry N. L. Avery, Karen Coulman, Jane M. Blazeby, on behalf of the ROMIO study group

Abstract

Core outcome set (COS) developers increasingly employ Delphi surveys to elicit stakeholders' opinions of which outcomes to measure and report in trials of a particular condition or intervention. Research outside of Delphi surveys and COS development demonstrates that question order can affect response rates and lead to 'context effects', where prior questions determine an item's meaning and influence responses. This study examined the impact of question order within a Delphi survey for a COS for oesophageal cancer surgery. A randomised controlled trial was nested within the Delphi survey. Patients and health professionals were randomised to receive a survey including clinical and patient-reported outcomes (PROs), where the PRO section appeared first or last. Participants rated (1-9) the importance of 68 items for inclusion in a COS (ratings 7-9 considered 'essential'). Analyses considered the impact of question order on: (1) survey response rates; (2) participants' responses; and (3) items retained at end of the survey. In total, 116 patients and 71 professionals returned completed surveys. Question order did not affect response rates among patients, but fewer professionals responded when clinical items appeared first (difference = 31.3%, 95% confidence interval [CI] = 13.6-48.9%, P = 0.001). Question order led to different context effects within patients and professionals. While patients rated clinical items highly, irrespective of question order, more PROs were rated essential when appearing last rather than first (difference = 23.7%, 95% CI = 10.5-40.8%). Among professionals, the greatest impact was on clinical items; a higher percentage rated essential when appearing last (difference = 11.6%, 95% CI = 0.0-23.3%). An interaction between question order and the percentage of PRO/clinical items rated essential was observed for patients (P = 0.025) but not professionals (P = 0.357). Items retained for further consideration at the end of the survey were dependent on question order, with discordant items (retained by one question order group only) observed in patients (18/68 [26%]) and professionals (20/68 [29%]). In the development of a COS, participants' ratings of potential outcomes within a Delphi survey depend on the context (order) in which the outcomes are asked, consequently impacting on the final COS. Initial piloting is recommended with consideration of the randomisation of items in the survey to reduce potential bias. The randomised controlled trial reported within this paper was nested within the development of a core outcome set to investigate processes in core outcome set development. Outcomes were not health-related and trial registration was not therefore applicable.

X Demographics

X Demographics

The data shown below were collected from the profiles of 32 X users who shared this research output. Click here to find out more about how the information was compiled.
Mendeley readers

Mendeley readers

The data shown below were compiled from readership statistics for 80 Mendeley readers of this research output. Click here to see the associated Mendeley record.

Geographical breakdown

Country Count As %
Unknown 80 100%

Demographic breakdown

Readers by professional status Count As %
Student > Master 9 11%
Researcher 8 10%
Student > Ph. D. Student 7 9%
Student > Bachelor 7 9%
Student > Doctoral Student 4 5%
Other 16 20%
Unknown 29 36%
Readers by discipline Count As %
Medicine and Dentistry 21 26%
Nursing and Health Professions 7 9%
Social Sciences 4 5%
Pharmacology, Toxicology and Pharmaceutical Science 4 5%
Psychology 3 4%
Other 9 11%
Unknown 32 40%