Click here to order
Executive Summary
Teacher quality matters. In fact, it is the most important school-related factor influencing student achievement. Moreover, teacher compensation represents a significant public investment: in 2002 alone, the United States invested $192 billion in teacher pay and benefits. Given the size of this investment, there is remarkably little research to guide such critical decisions as whom to hire, retain, and promote. In the absence of a strong, robust, and deep body of research, the debate in this field is largely ideological.
This analysis reviews a wide range of empirical studies that examine the impact of teacher characteristics on teacher effectiveness in order to draw conclusions about the extent to which these characteristics are, in fact, linked with teacher performance. Greater clarity on the empirical evidence can inform the wisdom of current practice, guide state efforts as they struggle with No Child Left Behind compliance regarding teacher quality, and provide direction for future teacher policy decisions. For example, developing an approach to policy that values different and multiple teacher characteristics based on the research evidence may prove promising. It is important to note that many personal characteristics important for a good teacher are not measured in the studies reviewed. The focus is on aspects of teacher background that can be translated into policy recommendations and incorporated into teaching practice.
The framework for this study includes five broad categories of measurable and policy-relevant indicators to organize the teacher characteristics assumed to reflect teacher quality. It is notable that findings for these characteristics frequently differ for teachers at the elementary school level and teachers at the high school level and that the body of research on the subject of teacher quality suggests that the context of teaching matters (e.g., differences in grade levels, subject areas, and student populations). A refined understanding of how teacher attributes affect their performance across these different teaching contexts can be helpful in determining the range of potentially effective policy options.
The highlights of the empirical evidence include:
Teacher experience
• Several studies have found a positive effect of experience on teacher effectiveness; specifically, the “learning by doing” effect is most obvious in the early years of teaching.
Teacher preparation programs and degrees
• Research suggests that the selectivity/prestige of the institution a teacher attended has a positive effect on student achievement, particularly at the secondary level. This may partially be a reflection of the cognitive ability of the teacher.
• Evidence suggests that teachers who have earned advanced degrees have a positive impact on high school mathematics and science achievement when the degrees earned were in these subjects.
• Evidence regarding the impact of advanced degrees at the elementary level is mixed.
Teacher certification
• Research has demonstrated a positive effect of certified teachers on high school mathematics achievement when the certification is in mathematics.
• Studies show little clear impact of emergency or alternative-route certification on student performance in either mathematics or science, as compared to teachers who acquire standard certification.
Teacher coursework
• Teacher coursework in both the subject area taught and pedagogy contributes to positive education outcomes.
• Pedagogical coursework seems to contribute to teacher effectiveness at all grade levels, particularly when coupled with content knowledge.
• The importance of content coursework is most pronounced at the high school level.
• While the studies on the field experience component of teacher education are not designed to reveal causal relationships, they suggest positive effects in terms of opportunity to learn the profession and reduced anxiety among new teachers.
Teachers’ own test scores
• Tests that assess the literacy levels or verbal abilities of teachers have been shown to be associated with higher levels of student achievement.
• Studies show the National Teachers Examination and other state-mandated tests of basic skills and/or teaching abilities are less consistent predictors of teacher performance.
Given that many dimensions of teacher characteristics matter—preparation in both pedagogic and subject content, credentials, experience, and test scores—the findings from the literature imply that there is no merit in large-scale elimination of all credentialing requirements. Nor are improvements in teacher quality likely to be realized through the status quo. Rather, teacher policies need to reflect the reality that teaching is a complex activity that is influenced by the many elements of teacher quality. Most of the research does not seek to capture interactions among the multiple dimensions of teacher quality, and as a result, there are major gaps in the research that still need to be explored. Nor does the research fully address evidence about teacher quality at the elementary and middle school levels, in subjects other than mathematics, or among different populations of students (such as high poverty, English language learners, or special education).
In opposition to those who propose to eliminate all requirements for entering the teaching profession, this analysis supports a judicious use of the research evidence on teacher characteristics and teacher effectiveness. The evidence indicates that neither an extreme centralized bureaucratization nor a complete deregulation of teacher requirements is a wise approach for improving teacher quality. What holds a great deal more promise is refining the policies and practices employed to build a qualified body of teachers in elementary schools, middle schools, and high schools; for disadvantaged, special needs, and advantaged students; and for math, science, languages, English, social studies, and the arts.
Education policy makers and administrators would be well served by recognizing the complexity of the issue and adopting multiple measures along many dimensions to support existing teachers and to attract and hire new, highly qualified teachers. The research suggests that investing in teachers can make a difference in student achievement. In order to implement needed policies associated with staffing every classroom—even the most challenging ones—with high-quality teachers, substantial and targeted investments must first be made in both teacher quality and education research.
Chapter 1
Introduction:
The policy and research context
Are qualified teachers really quality teachers? Likewise, are hiring and compensation policies that reward certain qualifications the equivalent of investing in teacher quality? Does hiring and retaining qualified teachers lead to improvements in student achievement? Researchers and policy makers agree that teacher quality is a pivotal policy issue in education reform, particularly given the proportion of education dollars devoted to teacher compensation coupled with the evidence that teachers are the most important school-related factor affecting student achievement. However, considerable disagreement surrounds what specific teacher attributes indicate quality and how to better invest resources to provide quality teachers for all students. This review examines empirical evidence on the relationship between teacher attributes and teacher effectiveness with the goal of informing federal, state, and local teacher policy.
The policy context
Education is the compilation and product of many and varied resources. Among these, teachers stand out as a key to realizing the high standards that are increasingly emphasized in schools and school systems across the country. Despite general agreement about the importance of high-quality teachers, researchers, practitioners, policy makers, and the public have been unable to reach a consensus about what specific qualities and characteristics make a good teacher. Even more concerning is the array of policy statements regarding teacher preparation that have been set forth in the face of volumes of inconclusive and inconsistent evidence about what teacher attributes really contribute to desired educational outcomes. Policy makers are left with questions surrounding what counts as a quality teacher—information that could be valuable in guiding policies regarding whom to hire, whom to reward, and how best to distribute teachers across schools and classrooms. Answers to these questions have potentially important implications for the efficiency and equity of public education.
The intense interest in teacher policy is motivated by several compelling factors. One factor relates to the high proportion of educational dollars devoted to teacher compensation. The single largest category of educational spending is devoted to the purchase of teacher time. A substantial portion of the 1999-2000 national investment in public education, which totaled over $360 billion, was used to employ almost 2.9 million teachers to educate more than 46 million public elementary and secondary students (National Center for Education Statistics 2000).1 Guthrie and Rothstein (1998) assert that teacher salaries account for at least 50% of typical school district expenditures. Further, in their analysis of spending in the New York City public school system, Speakman et al. (1996) found that over 41% of the total expenditures in this district were devoted to the salaries and benefits of instructional teachers. An additional 6% was spent on other instructional personnel such as substitutes and paraprofessionals. This high level of investment mirrors the general sentiment among policy makers, researchers, and the general public that teachers are perhaps the most valuable resource allocated to student education.
Further, the enhancement of teacher quality is likely to be quite costly. Increases in teacher salaries, incentives such as loan-forgiveness programs, heightened teacher preparation requirements, and other efforts to prepare, recruit, and retain high-quality teachers are all associated with substantial costs. These costs could be managed by targeting specific areas of need where teacher shortages are most pronounced, such as particular subject areas (e.g., mathematics and science), types of classrooms (e.g., special education), and geographic areas (e.g., urban settings). Nevertheless, a clear sense of which teacher attributes really lead to improved educational outcomes should guide these important investment decisions, particularly given the many competing policy options to enhance teacher quality, as well as other attractive education policy proposals. In a context of limited resources, difficult policy choices must be made, and solid evidence should be used to guide those decisions.
The willingness of policy makers and taxpayers to devote such a large proportion of education dollars to teachers highlights the undisputed importance of teachers in realizing educational goals. A number of researchers have argued that teacher quality is a powerful predictor of student performance. In her analysis of teacher preparation and student achievement across states, Darling-Hammond (2000) reports that “measures of teacher preparation and certification are by far the strongest correlates of student achievement in reading and mathematics, both before and after controlling for student poverty and language status.” She contends that measures of teacher quality are more strongly related to student achievement than other kinds of educational investments such as reduced class size, overall spending on education, and teacher salaries.2
In contrast to the approach used by Darling-Hammond, which equates teacher quality with specific qualifications, Rivkin, Hanushek, and Kain (1998) identify teacher quality in terms of student performance outcomes.3 Their research identifies teacher quality as the most important school-related factor influencing student achievement. They conclude from their analysis of 400,000 students in 3,000 schools that, while school quality is an important determinant of student achievement, the most important predictor is teacher quality. In comparison, class size, teacher education, and teacher experience play a small role.
Hanushek (1992) estimates that the difference between having a good teacher and having a bad teacher can exceed one grade-level equivalent in annual achievement growth. Likewise, Sanders (1998) and Sanders and Rivers (1996) argue that the single most important factor affecting student achievement is teachers, and the effects of teachers on student achievement are both additive and cumulative. Further, they contend that lower achieving students are the most likely to benefit from increases in teacher effectiveness. Taken together, these multiple sources of evidence—however different in nature—all conclude that quality teachers are a critical
determinant of student achievement. In the current policy climate of standards-based reform, these findings make a strong case for gaining a better understanding of what really accounts for these effects. In other words, what is teacher quality?
The resource-intensive nature of teachers coupled with the empirical evidence documenting the critical role of teacher quality in realizing student achievement implies that teacher policy is a promising avenue toward better realizing goals of efficiency, equity, and adequacy in public education. Indeed, recommendations for reforming the preparation of teachers have become commonplace in reports aimed at improving public education (Bush 1987). For instance, almost two decades ago in its call for improved teacher preparation, the National Commission on Excellence in Education (1983) stated that “teacher preparation programs are too heavily weighted with courses in educational methods at the expense of courses in subjects to be taught.” The Carnegie Foundation for the Advancement of Teaching recommended that teacher education programs require a 3.0 grade point average for admission, and that teachers complete courses in an academic-core subject in four years before spending a fifth year learning about education (Boyer 1983). Likewise, the Holmes Group (1986) advised that all major universities with substantial enrollments of preservice teachers (i.e., those who are preparing to enter the teaching profe
ssion but who are not yet classroom teachers) should adopt the four-year liberal arts baccalaureate as a prerequisite for acceptance into their teacher education programs. A decade later the National Commission on Teaching and America’s Future proposed major changes in teacher preparation and licensure, recommending that authority over these matters be shifted from public officials to professional organizations (NCTAF 1996).4
The recent federal education legislation, No Child Left Behind (NCLB), further underlines the importance of having a high-quality teacher in every classroom in every school. The Bush Administration’s proposal, which specifies what defines a “highly qualified” teacher, is based on the premise that teacher excellence is vital to realizing improved student achievement.5 This legislation, along with typical hiring and compensation systems, assumes that years of teaching experience, teacher certification, engagement in certain types of coursework, and performance on standardized assessments are indicators of high-quality teachers.6
The purpose of this analysis is to review existing empirical evidence to draw conclusions about the specific characteristics that are linked with teacher performance. Greater clarity on the empirical evidence regarding teacher quality can inform the wisdom of current practice, guide state efforts in the struggle with NCLB compliance regarding teachers, and provide direction for future teacher policy.
The research context
In the context of this intense activity surrounding teacher policy, it makes sense to turn to the existing evidence on which teacher attributes are related to teacher effectiveness in order to guide policy decisions about hiring, compensation, and distribution with respect to teachers. However, the literature on teacher quality and qualifications has typically been viewed as inconsistent and inconclusive. Much of this perception has been fueled by a set of analyses conducted by Eric Hanushek over the past two decades. In his meta-analysis of studies examining the impact of several key educational resources on student achievement, Hanushek (1981, 1986, 1996, 1997) concluded that there is no systematic relationship between educational inputs and student performance. For example, with respect to teacher characteristics, Hanushek (1997) identified 171 estimates related to the impact of “teacher education” on student performance. Of these, he reported that 9% were statistically significant and positive, 5% were statistically significant and negative, and 86% were statistically insignificant. In addition, Hanushek included 41 estimates of the impact of teacher test scores on student outcomes. Of these, 37% were statistically significant and positive, 10% were statistically significant and negative, and 54% were not statistically significant. Finally, of the 207 studies that estimate the effect of teacher experience, 29% of the estimates were statistically significant and positive, 5% were statistically significant and negative, and 66% were not statistically significant.
Hanushek’s conclusions that resources are not systematically related to outcomes has been hotly challenged by a number of other researchers with respect to his “vote-counting” methodology (Hedges, Laine, and Greenwald 1994a, 1994b; Greenwald, Hedges, and Laine 1996; Krueger 2002) and how he weighted (or didn’t weight) the studies (Krueger 2002). The work by Hedges, Laine, and Greenwald demonstrated that the use of more sophisticated meta-analytical techniques to analyze the same set of studies included in Hanushek’s review produced far more consistent and compelling findings regarding the effect of educational resources—including variables related to the quality and quantity of teachers—on student achievement. Krueger’s (2002) critique of Hanushek’s methodology centered on how the various studies were weighted in Hanushek’s analysis. Essentially, Hanushek labeled each estimate of an effect as a “study,” so that one article could have several estimates, or studies, that are factored into Hanushek’s count of positive, negative, or statistically insignificant (positive and negative) effects. Krueger argues that this approach weights the various studies by the number of different estimates of the effect of a particular variable they include. Further, he contends that studies that report negative or statistically insignificant findings are more likely to include more estimates than those that find statistically significant positive effects. Krueger’s re-analysis of the studies that Hanushek included on the effect of pupil-teacher ratio and the effect of per-pupil expenditures demonstrates that other approaches to weighting the studies lead to a more consistent and positive story about the effect of these resources on student achievement.
In addition to these criticisms, Hanushek’s analysis was limited to the education production function literature, i.e., studies examining how educational resources (inputs) are systematically transformed into educational outcomes (outputs). On one hand, this set of studies could be argued to be too inclusive in the sense that even those studies that simply included an educational resource as a control variable might be inappropriately considered (e.g., a study including both class size and per-pupil expenditures). On the other hand, the production function literature could be contested as too exclusive in the sense that other methodological approaches, particularly those that allow the researcher to focus on more refined measures of what teachers know and can do, can also make valuable contributions to what we know about the value of educational resources. In contrast to the work of Hanushek and others who have looked at specific subgroups of studies (see, for example, Mayer, Mullens, Moore, and Ralph 2000; Wayne and Youngs 2003; Whitehurst 2002), the literature review presented here represents an analysis of a wide variety of empirical studies examining the impact of teacher attributes on teacher performance.
The approach taken here is similar to that used by Wilson, Floden, and Ferrini-Mundy (2001) in their review of the research on teacher preparation conducted for the U.S. Department of Education. Empirical studies that conform to a variety of accepted methodological approaches and use a range of measures of teacher effectiveness are used to ascertain what existing evidence says about the relationship between teacher attributes and their performance. In addition, this approach pays close attention to a number of contextual factors (e.g., level of education, subject area, type of student) as a way of drawing conclusions across studies. Clearly, the context of teaching is important and may affect the impact of the teacher attributes considered in this analysis. In fact, when existing studies are considered as a whole (without breaking them down by contextual factors such as subject area or grade level), findings tend to be inconsistent across studies; context variables may help to explain the apparent inconsistency of the existing research. In other words, a particular teacher attribute (e.g., a subject-specific master’s degree) may be an important predictor of teacher effectiveness in some contexts (e.g., high school math), but may not matter at all or may even have a negative effect in other contexts (e.g., first-grade reading). This careful attention to the context of teaching, wherever possible, helps to tease out some effects that would otherwise go undetected in reviews that neglect to consider these factors. The goal of this study is to sort through the available evidence to draw conclusions about what matters, what has been studied but has not been shown to matter, and what has not been adequately studied.
In the face of such seemingly inco
nsistent and inconclusive evidence, policy makers are side-stepping the research (or relying only on those studies that support their positions) to move forward with teacher policies, often without the benefit of research to guide their efforts. However, research can, and should, play a role in these decisions. For instance, numerous measures of what a teacher knows and can do have been routinely assumed to be important (at least as indicated through hiring strategies, salary schedules, and teacher reform agendas). However, questions continue to persist about what exactly a quality teacher is. In other words, what
teacher characteristics have been found to predict teacher effectiveness? This is a fundamental question that must precede policy discussions concerning what kinds of teacher qualities and qualifications to promote in aspiring teachers, whom to recruit and hire, what factors to use in setting salary schedules, and how to distribute teachers across different types of schools and classrooms to achieve equity and adequacy goals. This analysis examines the existing empirical literature on the relationship between teacher attributes and their effectiveness with the goal of informing policy on investing in teacher quality.
The next chapter describes the methodology used to review the literature on the relationship between teacher characteristics and their performance, and the chapter that follows presents the findings from this literature review. The final chapter concludes with a discussion of the implications of these findings for future research and policy.
Endnotes
1. The 1999-2000 NCES information is based on projected or preliminary data.
2. Of course, to the degree that reduced class sizes, overall educational spending, and teacher salaries are related to teacher quality, these can be viewed as investments in teacher quality, albeit indirect.
3. Rivkin, Hanushek, and Kain (1998) identify teachers as a major determinant of student performance, but do not describe teacher quality in terms of specific qualifications and characteristics. They show strong, systematic differences in expected achievement gains related to different teachers using a variance-components model.
4. Some have challenged the degree to which research supports the recommendations made by NCFAF. See Ballou & Podgursky (1997 , 1999, 2000) and, for a rebuttal, Darling-Hammond (2000).
5. Some argue that the qualifications identified in the NCLB legislation are more reflective of a “minimally qualified teacher” than a “highly qualified teacher.”
6. In contrast to many of the policy recommendations for stricter teacher qualifications, the Abell Foundation has recently released a report calling for the elimination of statewide coursework and certification requirements for teachers in favor of more flexible professional requirements (Abell Foundation 2001). Likewise, Hess (2002) argues for the deregulation of teacher preparation.
Jennifer King Rice is the Economic Policy Institute Research Associate and an associate professor in the Department of Education Policy and Leadership at the University of Maryland.
