When to adjust alpha during multiple testing: a consideration of disjunction, conjunction, and individual testing.
Scientists often adjust their significance threshold (alpha level) during null hypothesis significance testing in order to take into account multiple testing and multiple comparisons. This alpha adjustment has become particularly relevant in the context of the replication crisis in science. The pres...
| Publicado en: | Synthese Vol. 199; no. 3/4; pp. 10969 - 11001 |
|---|---|
| Autor principal: | |
| Formato: | Artículo |
| Publicado: |
Springer Nature
Dec2021
|
| Materias: | |
| Acceso en línea: | Ver este registro en EBSCOhost |
| fields | @attributes: recordID: 1 pdfLink: plink: https://search.ebscohost.com/login.aspx?direct=true&db=hlh&AN=154096924&site=ehost-live header: @attributes: shortDbName: hlh uiTerm: 154096924 longDbName: Humanities International Complete uiTag: AN controlInfo: bkinfo: jinfo: jid: 00397857 4LI jtl: Synthese issn: 00397857 maglogo: N pubinfo: dt: Dec2021 vid: 199 iid: 3/4 pid: 237 pub: Springer Nature artinfo: ui: 154096924 10.1007/s11229-021-03276-4 ppf: 10969 ppct: 32 formats: fmt: – @attributes: type: T – @attributes: type: P size: 2.4MB tig: atl: When to adjust alpha during multiple testing: a consideration of disjunction, conjunction, and individual testing. aug: au: Rubin, Mark affil: School of Psychology, Behavioural Sciences Building, The University of Newcastle, 2308, Callaghan, NSW, Australia su: Null hypothesis Conformance testing Multiple comparisons (Statistics) False positive error Statistical hypothesis testing sug: subj: Null hypothesis Conformance testing Multiple comparisons (Statistics) False positive error Statistical hypothesis testing keyword: Experimentwise error Familywise error Multiple comparisons Multiple testing Simultaneous testing Type I error ab: Scientists often adjust their significance threshold (alpha level) during null hypothesis significance testing in order to take into account multiple testing and multiple comparisons. This alpha adjustment has become particularly relevant in the context of the replication crisis in science. The present article considers the conditions in which this alpha adjustment is appropriate and the conditions in which it is inappropriate. A distinction is drawn between three types of multiple testing: disjunction testing, conjunction testing, and individual testing. It is argued that alpha adjustment is only appropriate in the case of disjunction testing, in which at least one test result must be significant in order to reject the associated joint null hypothesis. Alpha adjustment is inappropriate in the case of conjunction testing, in which all relevant results must be significant in order to reject the joint null hypothesis. Alpha adjustment is also inappropriate in the case of individual testing, in which each individual result must be significant in order to reject each associated individual null hypothesis. The conditions under which each of these three types of multiple testing is warranted are examined. It is concluded that researchers should not automatically (mindlessly) assume that alpha adjustment is necessary during multiple testing. Illustrations are provided in relation to joint studywise hypotheses and joint multiway ANOVAwise hypotheses. pubtype: Academic Journal doctype: Article src: R language: English refInfo: copyright: @attributes: flag: Y custom: Synthese is a copyright of Springer, 2021. All Rights Reserved. item: Synthese holder: Springer Nature dt: @attributes: year: 2021 holdings: @attributes: islocal: N |
|---|