↓ Skip to main content

Improving prediction models with new markers: a comparison of updating strategies

Overview of attention for article published in BMC Medical Research Methodology, September 2016
Altmetric Badge

About this Attention Score

  • Average Attention Score compared to outputs of the same age
  • Average Attention Score compared to outputs of the same age and source

Mentioned by

twitter
3 tweeters

Citations

dimensions_citation
18 Dimensions

Readers on

mendeley
42 Mendeley
You are seeing a free-to-access but limited selection of the activity Altmetric has collected about this research output. Click here to find out more.
Title
Improving prediction models with new markers: a comparison of updating strategies
Published in
BMC Medical Research Methodology, September 2016
DOI 10.1186/s12874-016-0231-2
Pubmed ID
Authors

D. Nieboer, Y. Vergouwe, Danna P. Ankerst, Monique J. Roobol, Ewout W. Steyerberg

Abstract

New markers hold the promise of improving risk prediction for individual patients. We aimed to compare the performance of different strategies to extend a previously developed prediction model with a new marker. Our motivating example was the extension of a risk calculator for prostate cancer with a new marker that was available in a relatively small dataset. Performance of the strategies was also investigated in simulations. Development, marker and test sets with different sample sizes originating from the same underlying population were generated. A prediction model was fitted using logistic regression in the development set, extended using the marker set and validated in the test set. Extension strategies considered were re-estimating individual regression coefficients, updating of predictions using conditional likelihood ratios (LR) and imputation of marker values in the development set and subsequently fitting a model in the combined development and marker sets. Sample sizes considered for the development and marker set were 500 and 100, 500 and 500, and 100 and 500 patients. Discriminative ability of the extended models was quantified using the concordance statistic (c-statistic) and calibration was quantified using the calibration slope. All strategies led to extended models with increased discrimination (c-statistic increase from 0.75 to 0.80 in test sets). Strategies estimating a large number of parameters (re-estimation of all coefficients and updating using conditional LR) led to overfitting (calibration slope below 1). Parsimonious methods, limiting the number of coefficients to be re-estimated, or applying shrinkage after model revision, limited the amount of overfitting. Combining the development and marker set using imputation of missing marker values approach led to consistently good performing models in all scenarios. Similar results were observed in the motivating example. When the sample with the new marker information is small, parsimonious methods are required to prevent overfitting of a new prediction model. Combining all data with imputation of missing marker values is an attractive option, even if a relatively large marker data set is available.

Twitter Demographics

The data shown below were collected from the profiles of 3 tweeters who shared this research output. Click here to find out more about how the information was compiled.

Mendeley readers

The data shown below were compiled from readership statistics for 42 Mendeley readers of this research output. Click here to see the associated Mendeley record.

Geographical breakdown

Country Count As %
Unknown 42 100%

Demographic breakdown

Readers by professional status Count As %
Researcher 9 21%
Student > Ph. D. Student 7 17%
Student > Master 6 14%
Professor > Associate Professor 4 10%
Other 3 7%
Other 3 7%
Unknown 10 24%
Readers by discipline Count As %
Medicine and Dentistry 15 36%
Mathematics 4 10%
Nursing and Health Professions 3 7%
Biochemistry, Genetics and Molecular Biology 2 5%
Computer Science 2 5%
Other 5 12%
Unknown 11 26%

Attention Score in Context

This research output has an Altmetric Attention Score of 2. This is our high-level measure of the quality and quantity of online attention that it has received. This Attention Score, as well as the ranking and number of research outputs shown below, was calculated when the research output was last mentioned on 02 October 2016.
All research outputs
#14,861,841
of 22,889,074 outputs
Outputs from BMC Medical Research Methodology
#1,448
of 2,024 outputs
Outputs of similar age
#194,914
of 322,819 outputs
Outputs of similar age from BMC Medical Research Methodology
#27
of 46 outputs
Altmetric has tracked 22,889,074 research outputs across all sources so far. This one is in the 33rd percentile – i.e., 33% of other outputs scored the same or lower than it.
So far Altmetric has tracked 2,024 research outputs from this source. They typically receive a lot more attention than average, with a mean Attention Score of 10.1. This one is in the 25th percentile – i.e., 25% of its peers scored the same or lower than it.
Older research outputs will score higher simply because they've had more time to accumulate mentions. To account for age we can compare this Altmetric Attention Score to the 322,819 tracked outputs that were published within six weeks on either side of this one in any source. This one is in the 36th percentile – i.e., 36% of its contemporaries scored the same or lower than it.
We're also able to compare this research output to 46 others from the same source and published within six weeks on either side of this one. This one is in the 34th percentile – i.e., 34% of its contemporaries scored the same or lower than it.