Improving the Reliability of Peer Review Without a Gold Standard.
Peer review plays a crucial role in accreditation and credentialing processes as it can identify outliers and foster a peer learning approach, facilitating error analysis and knowledge sharing. However, traditional peer review methods may fall short in effectively addressing the interpretive variabi...
| Publicado en: | Journal of Digital Imaging Vol. 37; no. 2; pp. 489 - 504 |
|---|---|
| Autores principales: | , , , , , , |
| Formato: | pictorial research tables/charts Journal Article |
| Publicado: |
Springer Nature
Apr2024
|
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ccm&AN=177626018&site=ehost-live header: @attributes: shortDbName: ccm uiTerm: 177626018 longDbName: CINAHL Complete uiTag: AN controlInfo: bkinfo: dissinfo: jinfo: jid: 08971889 DOQ jtl: Journal of Digital Imaging issn: 08971889 maglogo: N pubinfo: dt: Apr2024 vid: 37 iid: 2 pid: 237 pub: Springer Nature place: New York, New York artinfo: ui: 177626018 177626018 177626018 10.1007/s10278-024-00971-9 177626018 ppf: 489 ppct: 15 formats: fmt: – @attributes: type: T – @attributes: type: P tig: atl: Improving the Reliability of Peer Review Without a Gold Standard. aug: au: Äijö, Tarmo Elgort, Daniel Becker, Murray Herzog, Richard Brown, Richard K. J. Odry, Benjamin L. Vianu, Ron affil: Covera Health, New York, NY, USA sug: subj: Reliability Peer Review Accreditation Credentialing Digital Imaging Learning Health Knowledge Diagnostic Errors Quality Improvement Human Radiologists Confidence Validity Computer Simulation Peer Group Learning Methods Descriptive Statistics ab: Peer review plays a crucial role in accreditation and credentialing processes as it can identify outliers and foster a peer learning approach, facilitating error analysis and knowledge sharing. However, traditional peer review methods may fall short in effectively addressing the interpretive variability among reviewing and primary reading radiologists, hindering scalability and effectiveness. Reducing this variability is key to enhancing the reliability of results and instilling confidence in the review process. In this paper, we propose a novel statistical approach called "Bayesian Inter-Reviewer Agreement Rate" (BIRAR) that integrates radiologist variability. By doing so, BIRAR aims to enhance the accuracy and consistency of peer review assessments, providing physicians involved in quality improvement and peer learning programs with valuable and reliable insights. A computer simulation was designed to assign predefined interpretive error rates to hypothetical interpreting and peer-reviewing radiologists. The Monte Carlo simulation then sampled (100 samples per experiment) the data that would be generated by peer reviews. The performances of BIRAR and four other peer review methods for measuring interpretive error rates were then evaluated, including a method that uses a gold standard diagnosis. Application of the BIRAR method resulted in 93% and 79% higher relative accuracy and 43% and 66% lower relative variability, compared to "Single/Standard" and "Majority Panel" peer review methods, respectively. Accuracy was defined by the median difference of Monte Carlo simulations between measured and pre-defined "actual" interpretive error rates. Variability was defined by the 95% CI around the median difference of Monte Carlo simulations between measured and pre-defined "actual" interpretive error rates. BIRAR is a practical and scalable peer review method that produces more accurate and less variable assessments of interpretive quality by accounting for variability within the group's radiologists, implicitly applying a standard derived from the level of consensus within the group across various types of interpretive findings. pubtype: Academic Journal doctype: pictorial research tables/charts Journal Article ougenre: Article language: English refInfo: holdings: @attributes: islocal: N |
|---|