EPI Research Associate Richard Rothstein posted this response on the National Journal Experts blog, on the topic of lessons learned from Michelle Rhee.
Michelle Rhee, regardless of her specific impact on D.C. students, has chosen to join New York City schools chancellor Joel Klein in a national campaign for an overly simplistic and, on balance, harmful attack on incompetent teachers as the single most important problem facing public education. Although the test score gains on which Mr. Klein stakes his reputation have been exposed as seriously inflated, he and Ms. Rhee nonetheless chose the eve of her departure as the occasion for a “manifesto” of their views, published last week in the Washington Post.
The Klein-Rhee manifesto asserts that the difficulty of removing incompetent teachers “has left our school districts impotent and, worse, has robbed millions of children of a real future.”
Klein and Rhee base this assertion on a claim that, “as President Obama has emphasized, the single most important factor determining whether students succeed in school is not the color of their skin or their ZIP code or even their parents’ income — it is the quality of their teacher.”
I have posted on the Economic Policy Institute website an analysis of this claim, showing that it is not an accurate reflection of the president’s view, nor is it accurate on its own terms. I won’t repeat all the details here, but invite readers who are interested to read the analysis in its entirety.
It has become conventional in educational policy discussion to assert that “research shows” that “teachers are the most important influence on student achievement.” There is, in fact, no serious research that shows any such thing. The assertion results from a careless glide from “teachers being the most important in-school influence,” to teachers being the most important influence overall. But because school effects on average levels of achievement are smaller than the effects of families and communities, even if teachers were the largest school effect, they would not be a very big portion of the overall effect. A child with an average teacher who comes from a literate, economically secure, and stable family environment will, on average, have better achievement than a child with a superior teacher but with none of these contextual advantages. Of course, some children from improverished backgrounds will outperform typical children from literate and secure backgrounds, but on average, the extent to which children come to school prepared to take advantage of what school has to offer is a more important predictor than what even the best school can do.
The overemphasis on the influence of teachers stems in part from careless interpretations of studies purporting to show that if an average low-income student had the very best teachers for several years in a row, that student’s test scores would be close to those of an average higher-income student. This interpretation, however, is not based on research regarding any actual students who had the best teachers for several years in a row. Rather, it is based on an unfounded assumption that single year gains, even under the best circumstances, can simply accumulate if multiplied for several years, without any fade-out, an assumption contradicted by other research. Further, even if it were true that improving the quality of teachers (for example, by a full standard deviation, which is what the claim usually assumes) could have this effect, it says nothing about the relative importance of teachers vs. contextual factors. To do that, you’d need to compare the effect of a one standard deviation improvement in teacher quality, holding socio-economic status constant, to the effect of a one standard deviation improvement in relative socio-economic status, holding teacher quality constant, something no researcher has attempted. Or, more simply, compare the effect of a $1,000 per pupil investment in performance pay for teachers, to the effect of a $1,000 per pupil investment in a health clinic or early childhood program. But no researcher has attempted this, either. (Note that a recent experiment, in which teachers whose students had higher test scores were given a very substantial bonus, found virtually no impact on student achievement from the performance pay.)
And there is not even serious research support for the proposition that teachers are the most important in-school influence. Existing research only compares teacher to class size or per-pupil spending effects, showing, for example, that average achievement is likely to be higher with a superior teacher in a large class than with an average teacher in a small class. But there is little or no research comparing teacher quality to a host of other school characteristics that are arguably more important than class size or per-pupil spending levels – for example, the quality of school leadership, the nature of the curriculum, or the collaborative culture of a school’s faculty.
And finally, there is not even serious research that measures the effect of teachers on the full range of achievement. What we have is only the effect of teachers on student scores on very low-quality standardized tests of basic skills in math and reading. With the stakes increasingly high on these tests, we can have less confidence that teachers who produce higher student scores are actually producing greater literacy and mathematical ability, rather than better test taking skills or non-generalizable proficiency on a limited range of topics on which the tests focus. We would, however, expect higher quality teachers to produce not only literacy and mathematical ability, but civic responsibility, historical knowledge and insight, scientific background, cooperative work skills, good character, and appreciation of the arts and music. Perhaps teachers whose students have high math and reading scores are also teachers who produce these other outcomes, but we have no evidence one way or the other. We do know, however, that schools serving disadvantaged children under pressure to meet test-defined accountability targets have been reducing time spent on other critical subject matter.
There is, indeed, a consensus of scholarly opinion that student test scores, even of the value-added variety, are not sufficiently accurate to be used as the major indicator of teacher quality. They can’t adequately control for the differences in student characteristics between classrooms, they can’t control for differences in other school characteristics, and they rarely reflect a sufficiently large sample size for reliability. I recently had the privilege of joining a distinguished group of economists and psychometricians who reviewed this evidence, producing a report entitled, Problems With the Use of Student Test Scores to Evaluate Teachers. Test scores, in context, can be one part of a holistic evaluation, but not the major part.
Of course, schools should do a better job of removing teachers who don’t add value to what children bring to their classrooms. This should be a part, but only a part, of an overall strategy to raise the average level of student achievement. But when Joel Klein and Michelle Rhee launch a national campaign to make this the single most important focus of school improvement, they not only demoralize good teachers by exaggerating their responsibility for student outcomes, but ignore many other potential areas for intervention that might have bigger impacts.
In addition to those mentioned above, these also include working, either directly or in collaboration with other community organizations, to provide higher-quality early childhood and out-of-school time experiences, as well as the health and nutrition that also influence student achievement.
My review of the Klein-Rhee manifesto notes that the biggest threat to student achievement in the current age is our unprecedented economic catastrophe and its effect on parents and their children’s ability to gain from higher-quality schools. Chancellors Klein and Rhee aim to close the black-white achievement gap, but this will not be possible when 15% of all black children have an unemployed parent, compared to 8.5% of white children. (If we also include children whose parents have become so discouraged that they have given up looking for work, and children whose parents are working part-time because they can’t find full-time work, we find that 37% of black children have an unemployed or underemployed parent compared to 23% of white children.) Over half of all black children have a parent who has either been unemployed or underemployed during the past year. Thirty-six percent of black children now live in poverty.
When Joel Klein and Michelle Rhee equate concern with this catastrophe with a belief that children’s ZIP codes affect their achievement, they trivialize a national disaster. School chancellors and superintendents cannot do anything themselves to influence employment opportunities for the parents of our most challenged children. But when national policymakers and politicians exhibit a casual attitude towards unemployment, claiming that fiscal deficits prevent them from taking any action to address it, school chancellors and superintendents are irresponsible if they assure these politicians that, in effect, their lack of concern doesn’t matter because we can solve the problem by firing bad teachers. In this way, Michelle Rhee’s simplistic parting shot as D.C. Chancellor will contribute to the impoverishment not only of our children but of our national debate.
