Detecting unit of analysis problems in nested designs: statistical power and Type I error rates of the F test for groups-within-treatments effects. - Kromrey, Jeffrey D.; Dickinson, Wendy B. (Educational & Psychological Measurement, 1996)
A study was conducted to provide educational researchers with empirical estimates of the Type 1 error rate and statistical power of the F test of the classrooms-within-treatments effect in the nested analysis of variance model. The findings revealed that the test for groups within treatments did not demonstrate adequate power, even at the most liberal alpha level examined, to detect intraclass correlation in the sample data. It is concluded that the procedure was not effective in supporting decisions to utilize observations of individual students as opposed to group means as the unit of analysis. Implications of these findings for the design of classroom research are discussed.