Logistic regression models for patient-level prediction based on massive observational data: Do we need all data?
Objective: Provide guidance on sample size considerations for developing predictive models by empirically establishing the adequate sample size, which balances the competing objectives of improving model performance and reducing model complexity as well as computational requirements.Materials and Me...
| Published in: | International Journal of Medical Informatics Vol. 163 |
|---|---|
| Main Authors: | , , , , |
| Format: | research Journal Article |
| Published: |
Elsevier B.V.
Jul2022
|
| Online Access: | View this record in EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=156901185&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 156901185 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 13865056 JR4 jtl: International Journal of Medical Informatics issn: 13865056 maglogo: N pubinfo: dt: Jul2022 vid: 163 pid: 467 pub: Elsevier B.V. place: New York, New York artinfo: ui: 156901185 156901185 NLM35429722 156901185 10.1016/j.ijmedinf.2022.104762 NLM35429722 156901185 ppct: 1 formats: tig: atl: Logistic regression models for patient-level prediction based on massive observational data: Do we need all data? aug: au: John, Luis H. Kors, Jan A. Reps, Jenna M. Ryan, Patrick B. Rijnbeek, Peter R. affil: Department of Medical Informatics, Erasmus University Medical Center, Rotterdam, the Netherlands sug: subj: Logistic Regression Prospective Studies Sample Size Scales ab: Objective: Provide guidance on sample size considerations for developing predictive models by empirically establishing the adequate sample size, which balances the competing objectives of improving model performance and reducing model complexity as well as computational requirements.Materials and Methods: We empirically assess the effect of sample size on prediction performance and model complexity by generating learning curves for 81 prediction problems (23 outcomes predicted in a depression cohort, 58 outcomes predicted in a hypertension cohort) in three large observational health databases, requiring training of 17,248 prediction models. The adequate sample size was defined as the sample size for which the performance of a model equalled the maximum model performance minus a small threshold value.Results: The adequate sample size achieves a median reduction of the number of observations of 9.5%, 37.3%, 58.5%, and 78.5% for the thresholds of 0.001, 0.005, 0.01, and 0.02, respectively. The median reduction of the number of predictors in the models was 8.6%, 32.2%, 48.2%, and 68.3% for the thresholds of 0.001, 0.005, 0.01, and 0.02, respectively.Discussion: Based on our results a conservative, yet significant, reduction in sample size and model complexity can be estimated for future prediction work. Though, if a researcher is willing to generate a learning curve a much larger reduction of the model complexity may be possible as suggested by a large outcome-dependent variability.Conclusion: Our results suggest that in most cases only a fraction of the available data was sufficient to produce a model close to the performance of one developed on the full data set, but with a substantially reduced model complexity. pubtype: Academic Journal doctype: research Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|