1703 North Beauregard St.
Alexandria, VA 22311-1714
Tel: 1-800-933-ASCD (2723)
8:00 a.m. to 6:00 p.m. eastern time, Monday through Friday
Local to the D.C. area: 1-703-578-9600, press 2
Toll-free from U.S. and Canada: 1-800-933-ASCD (2723), press 2
All other countries: (International Access Code) + 1-703-578-9600, press 2
by Robert J. Marzano
Table of Contents
Perhaps now more than ever the quotation from Charles Dickens's A Tale of Two Cities describes the position of public education: “It was the best of times, it was the worst of times.” Actually, given the criticisms of public education, some of those directly involved in K through 12 education might argue that the only relevant part is “it was the worst of times.” This book, however, is about possibility, specifically the possibility that K–12 education is on the brink of the best of times if we so choose. My premise is that if we follow the guidance offered from 35 years of research, we can enter an era of unprecedented effectiveness for the public practice of education—one in which the vast majority of schools can be highly effective in promoting student learning. As subsequent chapters detail, any school in the United States can operate at advanced levels of effectiveness—if it is willing to implement what is known about effective schooling. Before examining this possibility, let us consider the criticisms of U.S. education—the argument for the worst of times.
The history of public education, particularly during the 20th century, is rife with criticisms (Tyack, 1974; Tyack & Tobin, 1994). Indeed, the century began with a massive effort to improve K-12 schooling, which was spear-headed by the Carnegie Foundation for the Advancement of Teaching. One significant aspect of that reform effort was the establishment of the “Carnegie unit” as the uniform standard for defining academic achievement.
Criticisms of public education and their accompanying reform efforts flourished for the first five decades of the century. However, it is the criticisms and reform efforts of the second half of the century that most profoundly affect us today. The first of these was spawned by the launching of Sputnik in 1957. Shocked by this event, the U.S. public began to question the rigor and viability of our schools. Indeed, influential figures such as Admiral Hyman Rickover (1959) forwarded the position that public education was weakening the intellectual capacity of our students. Rickover's book, Education and Freedom, made direct links between the security of the nation and the quality of education.
In the 1960s there was no hiatus from the harsh criticisms of public education. In fact, the study that arguably produced the most concrete evidence of the failures or inadequacies of public education was conducted in that decade. It was in the context of President Johnson's “war on poverty” that the Civil Rights Act of 1964, a cornerstone of Johnson's initiative, specified that the Commissioner of Education should conduct a nationwide survey of the availability of educational opportunity. The effort mounted was impressive even by today's standards. More than 640,000 students in grades 1, 3, 6, 9, and 12 took achievement and aptitude tests and were categorized into six ethnic and cultural groups. Sixty thousand teachers in 4,000 schools completed questionnaires about their background and training. The resulting report, Equality in Educational Opportunity, was published in July 1966. Although the work of a team of researchers (Coleman, Campbell, Hobson, McPartland, Mood, Weinfield, & York, 1966), it has become known as the “Coleman report” in deference to its senior author, James Coleman. To say the least, the findings did not paint a flattering picture of public education:
Taking all of these results together, one implication stands above all: that schools bring little to bear on a child's achievement that is independent of his background and general social context; and that this very lack of an independent effect means that the inequalities imposed on children by their home, neighborhood, and peer environment are carried along to become the inequalities with which they confront life at the end of school. (p. 325)
The report had a profound impact on public perceptions of schooling in the United States (Madaus, Airasian, & Kellaghan, 1980; Madaus, Kellaghan, Rakow, & King, 1979). Specifically, it dealt a veritable deathblow to the belief that schools could overcome students' backgrounds. Perhaps the most publicized finding from the report was that schools account for only about 10 percent of the variance in student achievement—the other 90 percent is accounted for by student background characteristics.
The findings in the Coleman report were corroborated when Christopher Jencks and his colleagues published Inequality: A Reassessment of the Effects of Family and Schooling in America, which was based on a reanalysis of Coleman's data (Jencks et al., 1972). Among the findings articulated in the Jencks study were the following:
The conclusions stated and implied in the Coleman and Jencks studies painted a sobering picture of U.S. education. If schools have little chance of overcoming the influence of students' background characteristics, why put any energy into school reform?
Although the nation viewed public education poorly in the 1960s and 1970s, the 1980s were even darker times. As Peter Dow (1991) explains in his book Schoolhouse Politics: Lessons from the Sputnik Era:
In 1983 educators and the general public were treated to the largest outpouring of criticism of the nation's schools in history, eclipsing even the complaints of the early 1950s. Nearly fifty reports totaling more than six thousand pages voiced a new wave of national concern about the troubled state of American education. They spoke of the fragmented state of the school curriculum, the failure to define any coherent, accepted body of learning, the excessive emphasis on teaching isolated facts, and the lack of attention to higher order skills and concepts. They called for more individualism of instruction, the development of a closer relationship between teachers and students, and methods that encourage the active participation of the student in the learning process. (p. 243)
Again, a single report laid the foundation for the outpouring of criticism. Without a doubt, A Nation at Risk: The Imperative for Educational Reform, issued by the National Commission on Excellence in Education, was considered by some as proof that K–12 education had indeed devolved to a state of irreversible disrepair. The report noted that “the educational foundations of our society are presently being eroded by a rising tide of mediocrity that threatens our very future as a nation and a people” (National Commission on Excellence in Education, 1983, p. 5). To punctuate the importance of the message about public education, the report claimed that “we have, in effect, been committing an act of unthinking, unilateral disarmament” (p. 5).
The effects of the report were profound, due in no small part to the fact that it was perceived as the sanctioned opinion of the White House. As David Berliner and Bruce Biddle note in their book The Manufactured Crisis: Myths, Frauds, and the Attack on America's Public Schools (Berliner & Biddle, 1995):
...in 1983, amid much fanfare, the White House released an incendiary document highly critical of American education. Entitled A Nation at Risk, this work was prepared by a prestigious committee under the direction of then Secretary of Education Terrell Bell and was endorsed in a speech by President Ronald Reagan. (p. 3)
The effects of A Nation at Risk persisted through the 1990s. Indeed, some authors (Bennett, 1992; Finn, 1991) cite the report as one of the primary sources of evidence for public education's decline.
Although A Nation at Risk was sufficient to cast a negative shadow on education throughout the 1990s, a newer study, the Third International Mathematics and Science Study (TIMSS), was interpreted as evidence of the ineffectiveness of U.S. education. It involved a large-scale, cross-national comparison of the education systems in 41 countries. TIMSS researchers examined mathematics and science curricula, instructional practices, and school and social factors. In general, U.S. 4th grade students performed moderately well when compared to students of similar ages in other countries; 8th grade students less so; and 12th grade students performed quite poorly. Both technical reports of TIMSS (Schmidt, McKnight, & Raizen, 1996; U.S. Department of Education, National Center for Educational Statistics, 1998) and commentaries on TIMSS (Stevenson & Stigler, 1992; Stigler & Hiebert, 1999) interpret the results as evidence of a dire need for public education reform. Perhaps at the extreme, Chester Finn (1998), in a provocative article in the Wall Street Journal entitled “Why America Has the World's Dimmest Bright Kids,” described the findings in the following way:
Today the U.S. Department of Education officially releases the damning data, which come from the Third International Mathematics and Science Study, a set of tests administered to half a million youngsters in 41 countries in 1995. But the results have trickled out. We learned that our fourth-graders do pretty well compared with the rest of the world, and our eighth-graders' performance is middling to poor. Today we learn that our 12th-graders occupy the international cellar. And that's not even counting Asian lands like Singapore, Korea and Japan that trounced our kids in younger grades. They chose not to participate in this study. (p. A22)
Given the criticisms of public education that have flourished over the last half of the last century, it is clear that those who believe that it is the worst of times for public education have plenty of evidence for their position. Indeed, it is hard to imagine an argument for the position that it can be the best of times for public education.
My case for the position that public education is at the dawn of the best of times is not necessarily based on refuting the reports mentioned. Such arguments have been made for A Nation at Risk and, to some degree, TIMSS. Perhaps the most noteworthy of these arguments are found in David Berliner and Bruce Biddle's (1995) The Manufactured Crisis: Myths, Frauds, and the Attack on America's Public Schools and Gerald Bracey's (1997) Setting the Record Straight: Responses to Misconceptions about Public Education in the United States. These works take a rather aggressive stance that past research has been either misleading or misinterpreted to paint an unwarranted negative perspective of U.S. education. Although I do not share this view entirely, both works present compelling arguments and provide perspectives with which all educators should be familiar.
My basic position is quite simple: Schools can have a tremendous impact on student achievement if they follow the direction provided by the research. As evidence for this position, I will not use examples of specific schools mainly because other writers have already done so (see Darling-Hammond, 1997a; Reeves, 2002; Schmoker, 1999, 2001). Indeed, perhaps the most compelling evidence for this conclusion is the impressive list of schools that have “beat the odds” compiled by Education Trust (Barth et al., 1999). These high-poverty schools are referred to as “beat the odds” schools because they sport impressive academic achievement from students whose background characteristics would logically preclude it. Rather than present specific examples, I present evidence based on my attempts to synthesize the extant research over the last 35 years, which I assert has provided clear and unprecedented insight into the nature of schooling. I have presented technical and nontechnical descriptions of these efforts in several publications (Marzano, 1998a, 2000a; Marzano, Pickering, & Pollock, 2001). Although my case is made in detail in the chapters to come, it begins with three basic assertions.
The Coleman report was arguably the first high-visibility study of the second half of the 20th century to advance the position that schools have little impact on student achievement. Recall that its fundamental finding was that schools account for only about 10 percent of the variance in student achievement—a finding that was corroborated later by Jencks and colleagues (1972). Understanding the problems with using percentage of variance as the measure of a school's impact is the key to understanding how these findings could actually support the position that schools do make a difference. (For a technical discussion of issues regarding percentage of variance, see Technical Note 1, pp. 187–188.)
In nonstatistical terms, findings like those from the Coleman report are frequently interpreted in the following way: Assume you are examining the academic achievement of a group of 1,000 8th grade students who attend five different middle schools—200 in each school. Also assume that these students vary in their achievement scores—some have very high scores, some have very low scores, many have scores near the average. Taken at face value, the findings from the Coleman report imply that only about 10 percent of the differences in scores from student to student (more accurately, the squared differences) are a function of the quality of the schools these students attend. In other words, going to the best of the five schools as opposed to the worst of the five schools generates only about 10 percent of the differences in students' scores. What accounts for the other 90 percent of the differences in scores? Coleman and others (1966) concluded it is the background of the students.
How can these findings possibly be interpreted as evidence that schools can have a positive and significant influence on student achievement? Since the Coleman report was published, statisticians have found that using percentage of variance as an indication of a factor's importance is not the most useful way of interpreting research findings on academic achievement. In fact, as is the case with the Coleman report, this technique can paint an unnecessarily gloomy picture of a school's possible effects on student achievement.
Researchers Robert Rosenthal and Donald Rubin (1982) devised a more practical way to interpret research findings reported in terms of percentage of explained variance. Their approach is referred to as the Binomial Effect Size Display or BESD. (For a technical and more detailed explanation of the BESD, see Technical Note 2, pp. 189–190.) To illustrate Rosenthal and Rubin's BESD, consider Figure 1.1, which is based on Coleman's findings that schools account for only 10 percent of the variance in student achievement.
Percentage of Students Who Pass the Test
Percentage of Students Who Fail the Test
BESD = Binomial Effect Size Display
Although schools would be better described as representing many gradations of effectiveness from highly ineffective to highly effective, Rosenthal and Rubin's approach requires placing schools into one of those two broad categories. That is, a school is classified as being either effective or ineffective. Rosenthal and Rubin's approach also requires assuming that the students in the effective and the ineffective schools are given a test on which you would normally expect half of the students to pass and half to fail. Given these assumptions, we can now interpret Figure 1.1. The columns in Figure 1.1 are labeled “percentage of students who pass the test” and “percentage of students who fail the test.” In general, in the effective schools, 65.8 percent of students would pass the test, and only 34.2 percent would fail the test. Conversely, in general, in the ineffective schools only 34.2 percent of the students would pass the test, and 65.8 percent would fail it.
This perspective paints a far different picture of the findings from the Coleman report. In effective schools almost twice the percentage of students would pass the test (on which half are expected to fail and half to pass) than in the ineffective schools. The logical conclusion to draw from the Coleman report, then, is that effective schools do make a difference in student achievement.
The Coleman report and the Jencks follow-up study were the first in a series of studies to explore the impact of schools. Scores of similar studies have been conducted since. In a review of some of this research, Charles Teddlie, David Reynolds, and Pam Sammons (2000) indicate that many studies report that schools account for more variance in student achievement than Coleman's meager 10 percent. I have also synthesized much of that research (Marzano, 2000a). I analyzed the findings from 10 high-visibility studies (Bosker, 1992; Byrk & Raudenbush, 1992; Coleman et al., 1966; Creemers, 1994; Jencks et al., 1972; Luyten, 1994; Madaus et al., 1979; Rowe & Hill, 1994; Scheerens & Bosker, 1997; Stringfield & Teddlie, 1989) and discovered that the average finding was that schools account for 20 percent of the variance in student achievement—twice as much as that reported by Coleman. Why were the Coleman findings so low? George Madaus and his colleagues (1979) and Berliner and Biddle (1995) discussed this in detail. In brief, although Coleman and colleagues had access to student scores on standardized academic achievement tests, they chose to use a general measure of verbal ability (focused on vocabulary knowledge) as the primary outcome measure. This created a situation in which student background variables almost by definition were highly correlated with student achievement. Madaus and colleagues (1979) explain
. . . the construct “verbal ability” in the Coleman study has become equated with “school achievement” and the results have been generalized to the now popular myth that school facilities, resources, personnel, and curricula do not have a strong independent effect on achievement. Coleman's findings have been interpreted in the widest and most damaging sense....To assert that schools bring little influence to bear on a child's general verbal ability that is independent of his background and general social context is not the same as asserting that schools bring little influence to bear on pupils' achievement in a specific college preparatory physics course....The fact that home background variables seem to be vastly more influential in explaining verbal ability should not preclude or cloud any expectations we have that schools should have some independent effect on traditional curriculum areas which are systematically and explicitly treated as part of the instructional process. (p. 210)
The Coleman researchers' use of verbal ability as the primary dependent measure resulted in an underestimate of the effect of schools on student achievement.
How does the picture change if we use the updated estimate of 20 percent? To answer this question, we turn again to Rosenthal and Rubin's BESD approach in Figure 1.2 (p. 8).
As Figure 1.2 illustrates, the updated research indicates that effective schools generally have a fairly substantial impact on student achievement. Specifically, if a test on which you would normally expect half the students to pass and half the students to fail were given to students in effective schools, 72.4 percent of those students would pass the test and the remainder would fail. In the ineffective schools, however, only 27.6 percent of the students would pass the test. In the aggregate, then, the research indicates that schools, when run effectively, make a big difference in student achievement. Again, to quote Madaus and others (1979), the findings from studies that use appropriate student achievement measures “provide strong evidence for the differential effectiveness of schools; differences in school characteristics do contribute to differences in achievement.” (p. 223)
Assertions 1 & 2 are based on the convention of classifying schools into two broad and contrived categories—effective schools and ineffective schools. Given that there are about 92,000 public schools in the United States (National Center for Educational Statistics, 2002), we can assume that they approximate a normal distribution in terms of effectiveness, as depicted in Figure 1.3.
Let's consider those schools to the far right of the distribution in Figure 1.3—those schools at the 99th percentile in terms of their effectiveness. What effect do these schools have on students' achievement? Using the BESD approach, we find that 84.7 percent of the students in those schools would pass a test on which we would normally expect half the students to pass and half the students to fail. (The explanation for this is presented in Technical Note 3, p. 190). This would be true regardless of the background of the students who attend the school. Specifically, these schools provide interventions that are designed to overcome student background characteristics that might impede learning. These interventions are detailed in Section III of this book. For now, it is sufficient to say that this is a remarkable possibility—one that provides great hope for public education.
Research in the last 35 years demonstrates that effective schools can have a profound impact on student achievement. The remaining chapters articulate the guidelines provided by that research. Before articulating and discussing those guidelines, however, we must consider another perspective: Although the research provides clear guidance regarding effective schooling, is the U.S. public education system up to the challenge of following it?
In 1990 John Chubb and Terry Moe authored an influential book entitled Politics, Markets and America's Schools (Chubb & Moe, 1990). After conducting a study that involved more than 400 high schools and 10,000 high school teachers, Chubb and Moe reached some of the same conclusions that I have:
All things being equal, a student in an effectively organized school achieves at least a half-year more than a student in an ineffectively organized school over the last two years of high school. If this difference can be extrapolated to the normal four-year high school experience, an effectively organized school may increase the achievement of its students by more than one full year. That is a substantial school effect indeed. (p. 140)
Although this book asserts that public educators are up to the challenge of implementing what we know about effective schooling, Chubb and Moe assert that bureaucratic underpinnings of public schools doom to failure any attempts at school reform:
. . . we can only believe that the current “revolution” in American public education will prove a disappointment. It might have succeeded had it actually been a revolution, but it was not and was never intended to be, despite the lofty rhetoric. (p. 228)
They ultimately conclude that school choice (presumably in the form of vouchers) is the only viable way to implement the findings from the research.
Chubb and Moe offer compelling evidence. In brief, they demonstrate that the more district-level control or constraints put on a school, the lower the chances of the school being organized in an effective manner. According to Chubb and Moe, centralized control over personnel can be particularly debilitating to a school's effectiveness:
Among the reasons why direct external control may interfere with the development of an effective school, perhaps the most important is the potentially debilitating influence of external control over personnel. If principals have little or no control over who teaches in their schools, they are likely to be saddled with a number of teachers, perhaps even many teachers, whom they regard as bad fits. In an organization that works best through shared decisionmaking [sic] and delegated authority, a staff that is in conflict with the leader and with itself is a serious problem. ...Personnel policies that promote such conflict may be a school's greatest external burden. (p. 152)
It is a small step from here to the necessity of vouchers and charter schools. Much of Chubb and Moe's argument has been criticized as “ideologically driven” (Berliner & Biddle, 1995, p. 75) as opposed to objectively driven by research results, but I believe their point is well taken. In effect, we stand at a crossroads—will we implement the research-based guidelines to produce schools that don't just work but that work remarkably well? To do so requires a powerful commitment to change the status quo.
Following the categorization scheme used by many researchers (Carroll, 1963; Cotton, 1995; Creemers, 1994; Elberts & Stone, 1988; Goldstein, 1997; Raudenbush & Byrk, 1988; Raudenbush & Willms, 1995; Rowe, Hill & Holmes-Smith, 1995; Scheerens, 1992; Scheerens & Bosker, 1997; van der Werf, 1997; Walberg, 1984; Wright, Horn, & Sanders, 1997), I've organized the results of 35 years of research into three general factors that influence student academic achievement: (1) school-level factors, (2) teacher-level factors, and (3) student-level factors.
School-level factors are primarily a function of school policy and schoolwide decisions and initiatives (a guaranteed and viable curriculum, challenging goals and effective feedback, parent and community involvement, a safe and orderly environment, and collegiality and professionalism).
Teacher-level factors are primarily under the control of individual teachers (specific instructional strategies, classroom management techniques, and classroom curriculum design). Student-level factors are generally associated with student background (home environment, learned intelligence and background knowledge, and motivation). Figure 1.4 depicts this model.
Implicit in Figure 1.4 is the notion that the school (as opposed to the district) is the proper focus for reform. Indeed, this is a consistent conclusion in the research literature (Scheerens & Bosker, 1997; Reynolds & Teddlie, 2000; Wang, Haertel & Walberg, 1993). While I share Chubb and Moe's concern that district-level central administration can sometimes impede school reform, I believe that the current structure of public education is malleable enough to benefit from the changes recommended in this book.
In keeping with the organization depicted in Figure 1.4, this book is divided into the following major sections. Section I deals with the five school-level factors, Section II deals with the three teacher-level factors, and Section III deals with the three student-level factors. Finally, Section IV addresses how a school might use the information in the three previous sections to engage in substantive change.
Thirty-five years of research provides remarkably clear guidance as to the steps schools can take to be highly effective in enhancing student achievement. Although the guidance from the research is clear, researchers and the public continue to debate whether public education is up to the task of following it. Following the lead of other studies, I have organized the research into three broad categories: school-level factors, teacher-level factors, and student-level factors.
Copyright © 2003 by Association for Supervision and Curriculum Development. All rights reserved.
No part of this publication—including the drawings, graphs, illustrations, or chapters, except for brief quotations in
critical reviews or articles—may be reproduced or transmitted in any form or by any means, electronic or mechanical,
including photocopy, recording, or any information storage and retrieval system, without permission from ASCD.
Subscribe to ASCD Express, our twice-monthly e-mail newsletter, to have practical, actionable strategies and information delivered to your e-mail inbox twice a month.
ASCD respects intellectual property rights and adheres to the laws governing them. Learn more about our permissions policy and submit your request online.