Cover Image
QR Code

Better P-Curves: Making P-Curve Analysis More Robust To Errors, Fraud, and Ambitious P-Hacking, A Reply To Ulrich and Miller (2015).

When studies examine true effects, they generate right-skewed p-curves, distributions of statistically significant results with more low (.01 s) than high (.04 s) p values. What else can cause a right-skewed p-curve? First, we consider the possibility that researchers report only the smallest signif...

Full description

Published in:Journal of Experimental Psychology. General Vol. 144; no. 6; pp. 1146 - 1153
Main Authors: Simonsohn, Uri, Simmons, Joseph P., Nelson, Leif D.
Format: Article
Published: American Psychological Association Dec2015
Subjects:
Statistical errors
Computer hacking
Skewness (Probability theory)
Online Access:View this record in EBSCOhost
Summary:When studies examine true effects, they generate right-skewed p-curves, distributions of statistically significant results with more low (.01 s) than high (.04 s) p values. What else can cause a right-skewed p-curve? First, we consider the possibility that researchers report only the smallest significant p value (as conjectured by Ulrich & Miller, 2015), concluding that it is a very uncommon problem. We then consider more common problems, including (a) p-curvers selecting the wrong p values, (b) fake data, (c) honest errors, and (d) ambitiously p-hacked (beyond p < .05) results. We evaluate the impact of these common problems on the validity of p-curve analysis, and provide practical solutions that substantially increase its robustness.