p-Hacking by Post Hoc Selection With Multiple Opportunities: Detectability by Skewness Test?: Comment on Simonsohn, Nelson, and Simmons (2014).
Simonsohn, Nelson, and Simmons (2014) have suggested a novel test to detect p-hacking in research, that is, when researchers report excessive rates of "significant effects" that are truly false positives. Although this test is very useful for identifying true effects in some cases, it fails to ident...
| Published in: | Journal of Experimental Psychology. General Vol. 144; no. 6; pp. 1137 - 1146 |
|---|---|
| Main Authors: | , |
| Format: | Article |
| Published: |
American Psychological Association
Dec2015
|
| Subjects: | |
| Online Access: | View this record in EBSCOhost |
| Summary: | Simonsohn, Nelson, and Simmons (2014) have suggested a novel test to detect p-hacking in research, that is, when researchers report excessive rates of "significant effects" that are truly false positives. Although this test is very useful for identifying true effects in some cases, it fails to identify false positives in several situations when researchers conduct multiple statistical tests (e.g., reporting the most significant result). In these cases, p-curves are right-skewed, thereby mimicking the existence of real effects even if no effect is actually present. |
|---|