Testing the Difference of Correlated Agreement Coefficients for Statistical Significance.
This article addresses the problem of testing the difference between two correlated agreement coefficients for statistical significance. A number of authors have proposed methods for testing the difference between two correlated kappa coefficients, which require either the use of resampling methods...
| Publicado en: | Educational & Psychological Measurement Vol. 76; no. 4; pp. 609 - 638 |
|---|---|
| Autor principal: | |
| Formato: | Artículo |
| Publicado: |
Sage Publications Inc.
Aug2016
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=ssf&AN=116673800&site=ehost-live header: @attributes: shortDbName: ssf uiTerm: 116673800 longDbName: Social Sciences Full Text (H.W. Wilson) uiTag: AN controlInfo: bkinfo: jinfo: jid: 00131644 EPM jtl: Educational & Psychological Measurement issn: 00131644 maglogo: Y pubinfo: dt: Aug2016 vid: 76 iid: 4 pid: 344 pub: Sage Publications Inc. artinfo: ui: 116673800 10.1177/0013164415596420 ppf: 609 ppct: 29 formats: tig: atl: Testing the Difference of Correlated Agreement Coefficients for Statistical Significance. aug: au: Gwet, Kilem L. affil: 1Advanced Analytics, LLC, Gaithersburg, MD, USA su: Analysis of variance Confidence intervals Statistical correlation Research evaluation Statistics T-test (Statistics) Data analysis Inter-observer reliability Data analysis software Statistical models Descriptive statistics sug: subj: Analysis of variance Confidence intervals Statistical correlation Research evaluation Statistics T-test (Statistics) Data analysis Inter-observer reliability Data analysis software Statistical models Descriptive statistics keyword: agreement coefficients correlated agreement coefficients correlated kappas Gwet’s AC1 kappa significance test raters’ agreement testing correlated kappas agreement coefficients correlated agreement coefficients correlated kappas Gwet’s AC1 kappa significance test raters’ agreement testing correlated kappas ab: This article addresses the problem of testing the difference between two correlated agreement coefficients for statistical significance. A number of authors have proposed methods for testing the difference between two correlated kappa coefficients, which require either the use of resampling methods or the use of advanced statistical modeling techniques. In this article, we propose a technique similar to the classical pairwise t test for means, which is based on a large-sample linear approximation of the agreement coefficient. We illustrate the use of this technique with several known agreement coefficients including Cohen’s kappa, Gwet’s AC, Fleiss’s generalized kappa, Conger’s generalized kappa, Krippendorff’s alpha, and the Brenann–Prediger coefficient. The proposed method is very flexible, can accommodate several types of correlation structures between coefficients, and requires neither advanced statistical modeling skills nor considerable computer programming experience. The validity of this method is tested with a Monte Carlo simulation. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: N holdings: @attributes: islocal: N |
|---|