Screening nonrandomized studies for medical systematic reviews: a comparative study of classifiers.
Objectives: To investigate whether (1) machine learning classifiers can help identify nonrandomized studies eligible for full-text screening by systematic reviewers; (2) classifier performance varies with optimization; and (3) the number of citations to screen can be reduced.Methods: We used an open...
| Publicado en: | Artificial Intelligence in Medicine Vol. 55; no. 3; pp. 197 - 208 |
|---|---|
| Autores principales: | , , , |
| Formato: | research Journal Article |
| Publicado: |
Elsevier B.V.
2012 Jul
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=108132364&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 108132364 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 09333657 3HY jtl: Artificial Intelligence in Medicine issn: 09333657 maglogo: N pubinfo: dt: 2012 Jul vid: 55 iid: 3 pid: 1004 pub: Elsevier B.V. artinfo: ui: 108132364 108132364 NLM22677493 2011611656 10.1016/j.artmed.2012.05.002 NLM22677493 PMC3393813 108132364 ppf: 197 ppct: 11 formats: tig: atl: Screening nonrandomized studies for medical systematic reviews: a comparative study of classifiers. aug: au: Bekhuis T Demner-Fushman D Bekhuis, Tanja Demner-Fushman, Dina affil: Department of Biomedical Informatics, School of Medicine, University of Pittsburgh, Pittsburgh, PA 15232, USA sug: subj: Algorithms Artificial Intelligence Data Mining Methods Literature Research, Medical Classification Human Medical Informatics Probability Systematic Review ab: Objectives: To investigate whether (1) machine learning classifiers can help identify nonrandomized studies eligible for full-text screening by systematic reviewers; (2) classifier performance varies with optimization; and (3) the number of citations to screen can be reduced.Methods: We used an open-source, data-mining suite to process and classify biomedical citations that point to mostly nonrandomized studies from 2 systematic reviews. We built training and test sets for citation portions and compared classifier performance by considering the value of indexing, various feature sets, and optimization. We conducted our experiments in 2 phases. The design of phase I with no optimization was: 4 classifiers × 3 feature sets × 3 citation portions. Classifiers included k-nearest neighbor, naïve Bayes, complement naïve Bayes, and evolutionary support vector machine. Feature sets included bag of words, and 2- and 3-term n-grams. Citation portions included titles, titles and abstracts, and full citations with metadata. Phase II with optimization involved a subset of the classifiers, as well as features extracted from full citations, and full citations with overweighted titles. We optimized features and classifier parameters by manually setting information gain thresholds outside of a process for iterative grid optimization with 10-fold cross-validations. We independently tested models on data reserved for that purpose and statistically compared classifier performance on 2 types of feature sets. We estimated the number of citations needed to screen by reviewers during a second pass through a reduced set of citations.Results: In phase I, the evolutionary support vector machine returned the best recall for bag of words extracted from full citations; the best classifier with respect to overall performance was k-nearest neighbor. No classifier attained good enough recall for this task without optimization. In phase II, we boosted performance with optimization for evolutionary support vector machine and complement naïve Bayes classifiers. Generalization performance was better for the latter in the independent tests. For evolutionary support vector machine and complement naïve Bayes classifiers, the initial retrieval set was reduced by 46% and 35%, respectively.Conclusions: Machine learning classifiers can help identify nonrandomized studies eligible for full-text screening by systematic reviewers. Optimization can markedly improve performance of classifiers. However, generalizability varies with the classifier. The number of citations to screen during a second independent pass through the citations can be substantially reduced. pubtype: Academic Journal doctype: research Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|