↓ Skip to main content

The art and science of study identification: a comparative analysis of two systematic reviews

Overview of attention for article published in BMC Medical Research Methodology, February 2016
Altmetric Badge

About this Attention Score

  • In the top 5% of all research outputs scored by Altmetric
  • High Attention Score compared to outputs of the same age (93rd percentile)
  • High Attention Score compared to outputs of the same age and source (93rd percentile)

Mentioned by

twitter
54 X users

Citations

dimensions_citation
17 Dimensions

Readers on

mendeley
125 Mendeley
citeulike
1 CiteULike
You are seeing a free-to-access but limited selection of the activity Altmetric has collected about this research output. Click here to find out more.
Title
The art and science of study identification: a comparative analysis of two systematic reviews
Published in
BMC Medical Research Methodology, February 2016
DOI 10.1186/s12874-016-0118-2
Pubmed ID
Authors

Laura Rosen, Ruth Suhami

Abstract

Systematic reviews (SRs) form the foundation for guidelines and evidence-based policy in medicine and public health. Although similar systematic reviews may include non-identical sets of studies, and it is recognized that different sets of studies may lead to different conclusions, little work has been published on why SR study cohorts differ. We took advantage of concurrent publication of two SRs on the same topic - prevention of child exposure to tobacco smoke - to understand why study cohorts differed in the two reviews. We identified all studies included in just one review, investigated validity of specified reasons for exclusions, and, using database records, explored reasons for study non-identification. We assessed review methods and discordancy, and attempted to assess whether changes in study cohorts would have changed conclusions. Sixty-one studies were included in the two reviews. Thirty-five studies were present in just one review; of these, twenty were identified and excluded by the parallel review. Omissions were due to: review scope (9 studies, 26 %), outcomes of interest not measured (8 studies, 23 %), exclusion of reports with inadequate reporting (6 studies, 17 %), mixed or unclear reasons (3 studies, 8 %), search strategies concerning filters, tagging, and keywords (3 studies, 8 %), search strategies regarding sources (PUBMED not searched) (2 studies, 6 %); discordant interpretation of same eligibility criteria (2 studies, 6 %), and non-identification due to non-specific study topic (2 studies, 6 %). Review conclusions differed, but were likely due to differences in synthesis methods, not differences in study cohorts. The process of study identification for SRs is part art and part science. While some differences are due to differences in review scope, outcomes measured, or reporting practices, others are caused by search methods or discrepancies in reviewer interpretations. Different study cohorts may or may not be a cause of differing SR results. Completeness of SR study cohorts could be enhanced by 1 - independent identification of studies by at least two reviewers, as recommended by recent guidelines, 2 - searching PUBMED with free-text keywords in addition to MEDLINE to identify recent studies, and 3 - Using validated search filters.

X Demographics

X Demographics

The data shown below were collected from the profiles of 54 X users who shared this research output. Click here to find out more about how the information was compiled.
Mendeley readers

Mendeley readers

The data shown below were compiled from readership statistics for 125 Mendeley readers of this research output. Click here to see the associated Mendeley record.

Geographical breakdown

Country Count As %
Japan 1 <1%
United Kingdom 1 <1%
Denmark 1 <1%
Unknown 122 98%

Demographic breakdown

Readers by professional status Count As %
Student > Bachelor 44 35%
Researcher 12 10%
Librarian 10 8%
Student > Master 8 6%
Other 4 3%
Other 17 14%
Unknown 30 24%
Readers by discipline Count As %
Medicine and Dentistry 63 50%
Nursing and Health Professions 6 5%
Social Sciences 5 4%
Psychology 4 3%
Arts and Humanities 3 2%
Other 10 8%
Unknown 34 27%
Attention Score in Context

Attention Score in Context

This research output has an Altmetric Attention Score of 30. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 09 August 2019.
All research outputs
#1,195,689
of 23,954,688 outputs
Outputs from BMC Medical Research Methodology
#140
of 2,118 outputs
Outputs of similar age
#20,781
of 302,110 outputs
Outputs of similar age from BMC Medical Research Methodology
#3
of 33 outputs
Altmetric has tracked 23,954,688 research outputs across all sources so far. Compared to these this one has done particularly well and is in the 95th percentile: it's in the top 5% of all research outputs ever tracked by Altmetric.
So far Altmetric has tracked 2,118 research outputs from this source. They typically receive a lot more attention than average, with a mean Attention Score of 10.6. This one has done particularly well, scoring higher than 93% of its peers.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 302,110 tracked outputs that were published within six weeks on either side of this one in any source. This one has done particularly well, scoring higher than 93% of its contemporaries.
We're also able to compare this research output to 33 others from the same source and published within six weeks on either side of this one. This one has done particularly well, scoring higher than 93% of its contemporaries.