Source 5A and 5B: Graphics originally printed in TNTP's The Widget Effect: Our National Failure to Acknowledge and Act of
Differences in Teacher Effectiveness.29
There are important
differences in teacher effectiveness that are systematically related to observed teacher attributes.
Nevertheless, the teacher value - added scores computed in this study, despite reflecting
differences in teacher effectiveness, are vulnerable to bias.
The widget effect: Our national failure to acknowledge and act on
differences in teacher effectiveness.
In this study, we compare the teacher quality distributions in charter schools and traditional public schools, and examine mechanisms that might explain cross-sector
differences in teacher effectiveness as measured by teacher value - added scores using school and teacher level data from Florida.
The Obama administration's Race to the Top competition brought renewed attention to teacher evaluation, as did The New Teacher Project's 2009 landmark report, «The Widget Effect: Our National Failure to Acknowledge and Act on
Differences in Teacher Effectiveness.»
In 2009, The New Teacher Project (TNTP)'s The Widget Effect documented the failure to recognize and act on
differences in teacher effectiveness.
To what degree are year - to - year changes due to
differences in teacher effectiveness that schools can actually control?
A growing body of evidence suggests that there are dramatic
differences in teacher effectiveness not reflected in the subjective evaluations now in place.
For comparison, and to distinguish measurement error from true
differences in teacher effectiveness, the authors ran similar correlations with randomly separated groups of students.
States and districts can achieve four objectives related to this assurance by recognizing and acting upon
differences in teacher effectiveness:
More important, they find that
the differences in teacher effectiveness within pathways far exceed the average differences between pathways.
In fact, studies of informal surveys of principals (see «When Principals Rate Teachers,» research, Spring 2006) and teacher ratings by mentor teachers find that these more - subjective evaluation methods have similar power to detect
differences in teacher effectiveness as the TES ratings.
And recent studies that consider within - school
differences in teacher effectiveness show just how important teachers are (see Figure 1).
In short, research shows very large
differences in teacher effectiveness.
In other words, the observed teacher practices included in the TES evaluation system appear to capture a little less than half of the overall
differences in teacher effectiveness.
The Widget Effect: Our National Failure to Acknowledge and Act on
Differences in Teacher Effectiveness The New Teacher Project (TNTP), 2009 Extensive research of teacher evaluation systems in 12 schools districts highlights our pervasive and longstanding failure to recognize and respond to variations in the effectiveness of our teachers.
New York, NY — StudentsFirstNY today issued a brief analysis comparing
the difference in teacher effectiveness between New York City's high poverty and low poverty districts following the State Education Department's recent release of teacher evaluation data.
Duckworth attributes the difference to perseverance rather than talent: There wasn't any significant
difference in teacher effectiveness based on the SAT scores and college GPAs of the job applicants, she calculated.
The data are quite clear about a key issue:
The difference in teacher effectiveness is greater within these various routes (including TFA) than between them.
Not exact matches
In a 1956 review of the research on «School Personnel and Mental Health,» J. T. Hunt, a professor at the University of North Carolina, noted that «efforts to identify personality
differences between superior and inferior school personnel, to isolate a «
teacher personality,» or to predict either competence or
effectiveness of student
teachers by means of psychometric or projective instruments, led to limited results.»
As a result, wide
differences in teachers»
effectiveness persist over time.»
The study — conducted by William L. Sanders, the statistician who pioneered the concept of «value - added» analysis of teaching
effectiveness — found that there was basically no
difference in the achievement levels of students whose
teachers earned the prestigious NBPTS credential, those who tried but failed to earn it, those who never tried to get the certification, or those who earned it after the student...
Moreover, even small
differences in measured
effectiveness can have practical consequences for schools and
teachers, depending on how these assessments are used.
We calculate that districts would only have to lay off 132
teachers under an
effectiveness - based system
in order to achieve the same budgetary savings they would achieve with 145 layoff notices under today's seniority - driven system, a
difference of about 10 percent.
These within - school
differences likely understate the overall import of
teacher effectiveness because, as recent evidence suggests, there are also
differences in teacher quality across schools.
But since the report's publication, scholars have developed more precise data on
teacher effectiveness, and, by probing at
differences in teacher quality within schools, have found very large impacts of
teacher quality on student achievement.
The ubiquity of «satisfactory» ratings stands
in contrast to a rapidly growing body of research that examines
differences in teachers»
effectiveness at raising student achievement.
Simple common sense would dictate that the
effectiveness of different
teachers or schools should have as least as much to do with
differences in what is being taught as who is doing the teaching, or under whose roof and which assessment regime.
Yet as we embrace this piece of conventional wisdom, we must discard another: the widespread sentiment that there are large
differences in effectiveness between traditionally certified
teachers and uncertified or alternatively certified
teachers.
However, the strength of this preference depends on two things: the actual
difference in turnover rates and the
difference in effectiveness between an experienced and a novice
teacher.
The second study's authors, Donald Boyd, Pamela Grossman, Hamilton Lankford, Susanna Loeb, and James Wyckoff, also find few significant
differences in effectiveness between traditionally certified New York City
teachers and
teachers entering through alternative pathways, such as Teach For America or the New York City Teaching Fellows program.
Starting again with the estimates of the
difference in effectiveness of
teachers, it is possible to calculate the long - term economic impact of policies that would focus attention on the lowest - quality
teachers from U.S. classrooms.
To address questions about scalability, we will also explore whether there are
differences in the
effectiveness of PC based on the level of training and support provided to
teachers.
Our Texas results suggested there was little
difference in effectiveness between
teachers from various programs.
These
differences may originate
in collective bargaining agreements that make it relatively expensive for pilot schools to expand instructional hours and staffing and that favor
teacher seniority over classroom
effectiveness.
Unlike some other methods of estimating
teacher effectiveness, such as value - added modeling, MGP calculations do not try to adjust for
differences in student characteristics.
There is suggestive information
in the fact that there is not very much
difference in average
effectiveness by
teachers» routes into their careers (certified vs. non-certified).
If the
teacher's high value - added
in school A reflects her teaching ability, then the performance of students
in grade 4
in school B should go up by the
difference in the
effectiveness between her and the
teacher she is replacing.
Over the last decade, research
in public education has led us to three conclusions about the teaching profession:
teachers are the most important
in - school factor
in determining student achievement; there is wide variation
in teacher effectiveness; and those
differences really matter for kids.
In short, the education research community needs to prime the pump of evidence - based education with a supply of research findings that are of immediate relevance to workaday decision - making, e.g., recruiting tools that enhance the effectiveness of the workforce; ways to increase the productivity of the central office; and differences in the impact of available curriculum materials for particular types of teachers and student
In short, the education research community needs to prime the pump of evidence - based education with a supply of research findings that are of immediate relevance to workaday decision - making, e.g., recruiting tools that enhance the
effectiveness of the workforce; ways to increase the productivity of the central office; and
differences in the impact of available curriculum materials for particular types of teachers and student
in the impact of available curriculum materials for particular types of
teachers and students.
In contrast, no differences were seen across teacher effectiveness ratings in terms of providing explicit phonics instructio
In contrast, no
differences were seen across
teacher effectiveness ratings
in terms of providing explicit phonics instructio
in terms of providing explicit phonics instruction.
In contrast to statistically nonsignificant differences for the teachers within levels of school effectiveness, these statistically significant differences among teachers across schools suggest that a teacher's preferred style of interacting with students is a teaching dimension which is less well influenced by the practice of others at the school level than other dimensions of teaching being investigated in our study (e.g., time spent by students in independent reading, or degree of home communication
In contrast to statistically nonsignificant
differences for the
teachers within levels of school
effectiveness, these statistically significant
differences among
teachers across schools suggest that a
teacher's preferred style of interacting with students is a teaching dimension which is less well influenced by the practice of others at the school level than other dimensions of teaching being investigated
in our study (e.g., time spent by students in independent reading, or degree of home communication
in our study (e.g., time spent by students
in independent reading, or degree of home communication
in independent reading, or degree of home communication).
Embedded
in our larger
effectiveness study of ASMP was a small exploratory study, which involved listening to recordings of conversations between mentors and
teachers to see whether there was a
difference in the way the
teachers were being mentored.
While a coaching preference did not emerge as a general
difference among
teachers across school
effectiveness ratings, we did find that the practice of coaching during reading to provide word recognition instruction was found to be a characteristic of
teachers in the most effective schools and the most accomplished
teachers in general.
Although wide variation
in teacher effectiveness is well established, much less is known about
differences in teacher improvement over time.
The
difference in effectiveness between the average fifth - year
teacher compared to a rookie was more than nine times greater than the
difference between the average fifth year
teacher and those
in their 20th year.
A study of
teachers in New York City, for instance, concludes that the
difference between
teachers from programs that graduate
teachers of average
effectiveness and those whose
teachers are the most effective is roughly comparable to the (regression - adjusted) achievement
difference between students who are and are not eligible for subsidized lunch.
If we think about improvement as measuring the
difference between a
teacher's
effectiveness at the beginning of a period and her
effectiveness at the end, the change over time will be subject to errors
in both the starting and the ending value.
Instead, we find that
differences in returns to experience on
teacher productivity, which is significantly higher
in the charter sector, explains most of the observed cross-sector
effectiveness gaps.