Testing What Matters (If You Must Test at All): A Context-Driven Approach to Substantive and Statistical Significance.

For over a half century, various fields in the behavioral and social sciences have debated the appropriateness of null hypothesis significance testing (NHST) in the presentation and assessment of research results. A long list of criticisms has fueled the so-called significance testing controversy. T...

Full description

Bibliographic Details
Published in:American Journal of Political Science (John Wiley & Sons, Inc.) Vol. 59; no. 3; pp. 775 - 789
Main Author: Gross, Justin H.
Format: Article
Published: John Wiley & Sons, Inc. Jul2015
Subjects:
Online Access:View this record in EBSCOhost
Description
Summary:For over a half century, various fields in the behavioral and social sciences have debated the appropriateness of null hypothesis significance testing (NHST) in the presentation and assessment of research results. A long list of criticisms has fueled the so-called significance testing controversy. The conventional NHST framework encourages researchers to devote excessive attention to statistical significance while underemphasizing practical (e.g., scientific, substantive, social, political) significance. I introduce a simple, intuitive approach that grounds testing in subject-area expertise, balancing the dual concerns of detectability and importance. The proposed practical and statistical significance test allows the social scientist to test for real-world significance, taking into account both sampling error and an assessment of what parameter values should be deemed interesting, given theory. The matter of what constitutes practical significance is left in the hands of the researchers themselves, to be debated as a natural component of inference and interpretation.