↓ Skip to main content

Assessing imprecision in Cochrane systematic reviews: a comparison of GRADE and Trial Sequential Analysis

Overview of attention for article published in Systematic Reviews, July 2018
Altmetric Badge

About this Attention Score

  • In the top 25% of all research outputs scored by Altmetric
  • High Attention Score compared to outputs of the same age (88th percentile)
  • High Attention Score compared to outputs of the same age and source (80th percentile)

Mentioned by

blogs
1 blog
twitter
24 X users

Citations

dimensions_citation
118 Dimensions

Readers on

mendeley
58 Mendeley
You are seeing a free-to-access but limited selection of the activity Altmetric has collected about this research output. Click here to find out more.
Title
Assessing imprecision in Cochrane systematic reviews: a comparison of GRADE and Trial Sequential Analysis
Published in
Systematic Reviews, July 2018
DOI 10.1186/s13643-018-0770-1
Pubmed ID
Authors

Greta Castellini, Matteo Bruschettini, Silvia Gianola, Christian Gluud, Lorenzo Moja

Abstract

The evaluation of imprecision is a key dimension of the grading of the confidence in the estimate. Grading of Recommendations Assessment, Development and Evaluation (GRADE) gives recommendations on how to downgrade evidence for imprecision, but authors vary in their use. Trial Sequential Analysis (TSA) has been advocated for a more reliable assessment of imprecision. We aimed to evaluate reporting of and adherence to GRADE and to compare the assessment of imprecision of intervention effects assessed by GRADE and TSA in Cochrane systematic reviews. In this cross-sectional study, we included 100 Cochrane reviews irrespective of type of intervention with a key dichotomous outcome meta-analyzed and assessed by GRADE. The methods and results sections of each review were assessed for adequacy of imprecision evaluation. We re-analyzed imprecision following the GRADE Handbook and the TSA Manual. Overall, only 13.0% of reviews stated the criteria they applied to assess imprecision. The most common dimensions were the 95% width of the confidence intervals and the optimal information size. Review authors downgraded 48.0% of key outcomes due to imprecision. When imprecision was re-analyzed following the GRADE Handbook, 64% of outcomes were downgraded. Agreement between review authors' assessment and assessment by the authors of this study was moderate (kappa 0.43, 95% confidence interval [CI] 0.23 to 0.58). TSA downgraded 69.0% outcomes due to imprecision. Agreement between review authors' GRADE assessment and TSA, irrespective of downgrading levels, was moderate (kappa 0.43, 95% CI 0.21 to 0.57). Agreement between our GRADE assessment following the Handbook and TSA was substantial (kappa 0.66, 95% CI 0.49 to 0.79). In a sample of Cochrane reviews, methods for assessing imprecision were rarely reported. GRADE according to Handbook guidelines and TSA led to more severe judgment of imprecision rather than GRADE adopted by reviews' authors. Cochrane initiatives to improve adherence to GRADE Handbook are warranted. TSA may transparently assist in such development.

X Demographics

X Demographics

The data shown below were collected from the profiles of 24 X users who shared this research output. Click here to find out more about how the information was compiled.
Mendeley readers

Mendeley readers

The data shown below were compiled from readership statistics for 58 Mendeley readers of this research output. Click here to see the associated Mendeley record.

Geographical breakdown

Country Count As %
Unknown 58 100%

Demographic breakdown

Readers by professional status Count As %
Student > Master 14 24%
Student > Bachelor 6 10%
Other 6 10%
Student > Postgraduate 5 9%
Student > Ph. D. Student 4 7%
Other 8 14%
Unknown 15 26%
Readers by discipline Count As %
Medicine and Dentistry 20 34%
Nursing and Health Professions 10 17%
Neuroscience 2 3%
Business, Management and Accounting 1 2%
Agricultural and Biological Sciences 1 2%
Other 4 7%
Unknown 20 34%
Attention Score in Context

Attention Score in Context

This research output has an Altmetric Attention Score of 19. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 01 March 2023.
All research outputs
#1,913,009
of 25,218,929 outputs
Outputs from Systematic Reviews
#304
of 2,208 outputs
Outputs of similar age
#38,368
of 336,416 outputs
Outputs of similar age from Systematic Reviews
#11
of 52 outputs
Altmetric has tracked 25,218,929 research outputs across all sources so far. Compared to these this one has done particularly well and is in the 92nd percentile: it's in the top 10% of all research outputs ever tracked by Altmetric.
So far Altmetric has tracked 2,208 research outputs from this source. They typically receive a lot more attention than average, with a mean Attention Score of 13.1. This one has done well, scoring higher than 86% of its peers.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 336,416 tracked outputs that were published within six weeks on either side of this one in any source. This one has done well, scoring higher than 88% of its contemporaries.
We're also able to compare this research output to 52 others from the same source and published within six weeks on either side of this one. This one has done well, scoring higher than 80% of its contemporaries.