I've written several times recently about how short term gains in test scores are not associated with improved
later life outcomes for students.
I've written previously about the disconnect between near - term test score gains and changes
in later life outcomes.
To be sure, studies do not show a perfect one - to - one relationship between impacts on test scores and impacts
on later life outcomes — no one expects they would.
What evidence do we actually have to support the assumption that changing test scores is a reliable indicator of
changing later life outcomes?
A new study examines the connection between teacher reports about behavior when students are 11 and
later life outcomes for those students.
And we know that treatment group students are in a program that has previously demonstrated large advantages
in later life outcomes.
A growing literature establishes that high quality early childhood interventions targeted toward disadvantaged children have substantial impacts
on later life outcomes.
They find that non-response is predictive in six different national longitudinal data sets
of later life outcomes for students, including attainment, employment, and earnings, even after controlling for other relevant factors including cognitive ability.
Frequent, continuous and progressive learning outside the classroom is one of the most effective ways of
improving later life outcomes for all pupils, especially disadvantaged pupils who may otherwise have very limited horizons.
You mention the Project Star study showing that test scores in kindergarten correlated with
later life outcomes as proof that test scores are reliable indicators of school or program quality.
If we can't reliably use rigorously identified test score gains to
predict later life outcomes, then on what basis will regulators be able to judge quality to protect families against making bad choices?
People have been very slow to accept the fact that test scores are only weakly correlated
with later life outcomes because it would be so convenient if readily available and relatively inexpensive test scores could capture something as complex as school quality.
This same disconnect between test scores and
later life outcomes exists in several rigorously conducted studies of charter schools, including those of the Harlem Promise Academy, KIPP, High Tech High, SEED boarding charter schools, and no excuses charters in Texas.
If increasing test scores is a good indicator of
improving later life outcomes, we should see roughly the same direction and magnitude in changes of scores and later outcomes in most rigorously identified studies.
Test scores are useful but are not strong enough predictors
of later life outcomes to determine which are the «quality» schools that should be among the options available to families.
Meanwhile the unfashionable private choice schools and Mom and Pop charters seem to do much better
on later life outcomes than at changing test scores.
In examining the relative effectiveness of prospective and retrospective measures to
predict later life outcomes, we find that retrospective reports of the family environment most validly capture influences on the child in domains of strong emotional content but are less successful in cognitive domains.
Well, I've been making the argument for a while now that there is remarkably little evidence linking near - term changes in test scores to changes
in later life outcomes for students, like graduating high school, enrolling in college, completing college, and earnings.
A new study led by Nicholas W. Papageorge at Johns Hopkins University and IZA examines the connection in Great Britain between teacher reports about behavior when students are 11 and
later life outcomes for those students.
I'm not saying we never see a connection between changing test scores and
changing later life outcomes (e.g. Chetty, et al); I'm just saying that we do not regularly see that relationship.
The researchers» findings led to important insights into the links between young children's willpower and
later life outcomes, and into methods for enhancing self - control.
This concern is similar to issues that have arisen in other fields about the reliability of near - term indicators as proxies for
later life outcomes.
To illustrate the un-reliability of test score changes, I'm going to focus on rigorously identified research on school choice programs where we have
later life outcomes.
The fact that we can find a disconnect between test score changes and
later life outcomes in any literature, let alone in several, should undermine our confidence in test scores as a reliable indicator.
We don't really care about test scores per se, we care about them because we think they are near - term proxies for
later life outcomes that we really do care about — like graduating from high school, going to college, getting a job, earning a good living, staying out of jail, etc...
So, I think almost every credible researcher would agree that the vast majority of ways in which test scores are used by policymakers, regulators, portfolio managers, foundation officials, and other policy elites can not be reliable indicators of the ability of schools or programs to improve
later life outcomes.
Other than the general disconnect between test scores and
later life outcomes (in both directions), I notice that the No Excuses charter model that is currently the darling of the ed reform movement and that New York Times columnists have declared as the only type of «Schools that Work» tend not to fare nearly as well in later outcomes as they do on test scores.
Albert demonstrates that teacher character influences student character and that student character is predictive of
later life outcomes.
Essentially, this paper validates that item non-response is a useful proxy for character skills (probably conscientiousness) and is predictive of
later life outcomes.
Albert uses the Longitudinal Study of American Youth and confirms that student non-response on surveys is predictive of
later life outcomes.
If you like that paper, you'll love a new paper by Albert Cheng in which he looks at how teachers may affect student conscientiousness and
later life outcomes.
Test - based accountability proponents can point to research by Raj Chetty and colleagues that shows a connection between improvements in test scores and improved outcomes in adulthood, but their work examines testing from the 1980s, prior to the high - stakes era, and therefore does not capture how the threat of consequences might distort the relationship between test - score changes and
later life outcomes.
And the near - term test scores appear not to be very good proxies for
later life outcomes.
We already know from rigorous research that the program improves
later life outcomes, so I don't think we should be particularly troubled by these test results.
That is, the true test of the predictive power of «noncog» measures is not whether they are correlated with cognitive measures (like achievement scores), but whether they are correlated with
later life outcomes.
And even in the imaginary world in which VAM is used, learning growth on math and reading tests only captures a narrow portion of school quality, which is why those measures are not consistent predictors of
later life outcomes, like graduation, college attendance, and earnings.
But we should remember that 42 % of that control group are in the types of charter schools that other research has shown can produce giant test score gains without yielding much in
later life outcomes.
While of course we would generally like to see both test score gains and improved
later life outcomes, the thing we really care about is the later life outcomes.
Given that short - term gain in math and reading achievement are only weakly related to
later life outcomes, while a broad education that includes the arts and culturally enriching activities may be associated with long - term success, education reformers should wonder whether they are simply rediscovering what most Americans already know — «the arts are part of a well - rounded education for K - 12 students.»
«Learning by Doing», a 2015 report by DEMOS, comments that evidence suggests that character attributes not only reinforce academic learning but also have a significantly positive influence on
later life outcomes, including those relating to health, well - being and careers.
Recent evidence also shows that exposure to disruptive peers during elementary school worsens student achievement and
later life outcomes, including high school achievement, college enrollment, and earnings (see «Domino Effect,» research, Summer 2009).
-LSB-...] especially so given that the No Excuses charter model that has become the darling of ed reformers often comes up short at improving
later life outcomes, while private school choice programs seem to fare better at improving high school graduation, -LSB-...]