After this yearlong effort, the authors found stark
statistical differences between the schools that had participated versus those that hadn't.
Not exact matches
Translation: many of the
schools are given a ranked number, even though in a good deal of cases there is no
statistical basis for the
difference between one EMBA program and another.
The researchers also point out there were 1290 unique
school and grade combinations in the study sample — an average of 40 students per combination — which meant it «lacked
statistical power to find significant
differences between treatment conditions or grade levels».
One very recent study, using sophisticated
statistical techniques to summarize dozens of analyses across many states and cities, found that charter
schools generally outperform traditional public
schools in math, with little
difference between the two sectors in reading.
This is the exactly what would happen with the
statistical phenomenon of «regression towards the mean» — it indicates a serious flaw in the data analysis and interpretation, and suggests that there is no real
difference between the two types of
schools.
However, there are analytic and
statistical strategies that enable you to control for these
differences, that allow you to better isolate the true relationship
between school choice and student achievement.
The authors assess how different covariates contribute to improving the
statistical power of a randomization design and examine
differences between math and reading tests;
differences between test types (curriculum - referenced tests versus norm - referenced tests); and
differences between elementary
school and secondary
school, to see if the test subject, test type, or grade level makes a large
difference in the crucial design parameters.
The internal consistency of the CBCL in our sample, specifically the
School subscale, was somewhat low, reducing
statistical power (Bacon, 2004); however, our sample size and matched design provided enough power to uncover as statistically significant even small
differences between the groups on this measure.