Large-Sample Variance of Fleiss Generalized Kappa.
Cohen's kappa coefficient was originally proposed for two raters only, and it later extended to an arbitrarily large number of raters to become what is known as Fleiss' generalized kappa. Fleiss' generalized kappa and its large-sample variance are still widely used by researchers and were implemente...
| Publicado en: | Educational & Psychological Measurement Vol. 81; no. 4; pp. 781 - 791 |
|---|---|
| Autor principal: | |
| Formato: | Artículo |
| Publicado: |
Sage Publications Inc.
Aug2021
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ssf&AN=151158927&site=ehost-live header: @attributes: shortDbName: ssf uiTerm: 151158927 longDbName: Social Sciences Full Text (H.W. Wilson) uiTag: AN controlInfo: bkinfo: jinfo: jid: 00131644 EPM jtl: Educational & Psychological Measurement issn: 00131644 maglogo: Y pubinfo: dt: Aug2021 vid: 81 iid: 4 pid: 344 pub: Sage Publications Inc. artinfo: ui: 151158927 10.1177/0013164420973080 ppf: 781 ppct: 10 formats: tig: atl: Large-Sample Variance of Fleiss Generalized Kappa. aug: au: Gwet, Kilem L. affil: AgreeStat Analytics, Gaithersburg, MD, USA su: Statistics Sample size (Statistics) Confidence intervals Inter-observer reliability Sampling errors Research bias Data analysis software Statistical models sug: subj: Statistics Sample size (Statistics) Confidence intervals Inter-observer reliability Sampling errors Research bias Data analysis software Statistical models keyword: Cohen kappa Fleiss kappa Gwet AC1 interrater reliability Cohen kappa Fleiss kappa Gwet AC1 interrater reliability ab: Cohen's kappa coefficient was originally proposed for two raters only, and it later extended to an arbitrarily large number of raters to become what is known as Fleiss' generalized kappa. Fleiss' generalized kappa and its large-sample variance are still widely used by researchers and were implemented in several software packages, including, among others, SPSS and the R package "rel." The purpose of this article is to show that the large-sample variance of Fleiss' generalized kappa is systematically being misused, is invalid as a precision measure for kappa, and cannot be used for constructing confidence intervals. A general-purpose variance expression is proposed, which can be used in any statistical inference procedure. A Monte-Carlo experiment is presented, showing the validity of the new variance estimation procedure. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: N holdings: @attributes: islocal: N |
|---|