1703 North Beauregard St.
Alexandria, VA 22311-1714
Tel: 1-800-933-ASCD (2723)
8:00 a.m. to 6:00 p.m. eastern time, Monday through Friday
Local to the D.C. area: 1-703-578-9600, press 2
Toll-free from U.S. and Canada: 1-800-933-ASCD (2723), press 2
All other countries: (International Access Code) + 1-703-578-9600, press 2
February 2004 | Volume 61 | Number 5
Improving Achievement in Math and Science
James W. Stigler and James Hiebert
The TIMSS video studies provide a picture of what happens in mathematics classrooms in the United States and in other countries.
How does mathematics instruction differ from country to country? What do these international comparisons tell us about how to improve mathematics achievement?
We have been working for 10 years on a research program aimed at answering these questions. The TIMSS video studies document typical teaching practices in various countries. These studies employ the video survey, a novel methodology that combines two research traditions: qualitative classroom research and large-scale survey research. The video studies capture close-up pictures of the classroom processes used by national samples of 8th grade mathematics teachers in different countries. These teachers are not necessarily experienced or effective. They are ordinary teachers, teaching lessons that they routinely teach.
Why would we want to study a random sample of ordinary lessons? First, these lessons together represent what average teaching looks like in different countries. If we want to improve student learning, we must find a way to improve teaching in the average classroom. Even slight improvements in the average can positively affect millions of students. This concept represents a new way to formulate the question of how to improve teaching.
Second, studying a national sample of classroom lessons can help us discover whether policy initiatives have influenced classroom practice. All reform efforts to improve teaching and learning must pass through a final common pathway: the classroom. Most reforms get stopped short at the classroom door; all available evidence suggests that classroom practice has changed little in the past 100 years.
Finally, studying lessons from different cultures gives researchers and teachers the opportunity to discover alternative ideas about how we can teach mathematics. Watching lessons from other countries prompts questions about the assumptions that guide common practices in our own country. It is often a startling experience to journey back and forth, looking first at foreign videos and then back at our own.
The TIMSS 1995 video study (Stigler & Hiebert, 1999) examined national samples of 8th grade mathematics lessons from three countries: Germany, Japan, and the United States. Several findings from the first study provide important background information.
The lack of a shared language for describing teaching makes it very difficult to generate and disseminate professional knowledge. Even before beginning the video studies, we suspected that this problem existed; indeed, that suspicion was one of the reasons we chose to document teaching through videotapes instead of questionnaires. As the tapes started to arrive and we discussed what we saw on them, it became obvious that different people saw different things and described what they saw in different ways.
For example, we tried in the first study to mark where on the video each mathematics problem started and where it ended, a process that we thought might simplify our task by enabling us to analyze each problem separately. We could not agree on what a problem was (although we did manage to do so in the later study). Some observers would only count an activity as a problem if it involved students in sustained thinking over a long period of time. Others might count as a problem a brief exercise that students could solve quickly by recalling a solution that they had previously been taught. The word “problem” clearly means different things to different people. Other words and phrases, such as “develop concepts” or “teach for understanding,” pose similar challenges.
In part because we lack a shared language, attempts by policymakers to change what happens in classrooms often achieve either no results or unintended results as reform efforts get filtered through the weak communication channels we rely on to disseminate policy (Elmore, 2000). In our first video study, we asked teachers whether they had read mathematics education reform documents (for example, those published by the National Council of Teachers of Mathematics) and whether they implemented the documents' recommendations in their classrooms. Most teachers said that they had read such documents and that they used the reform ideas in their classrooms. However, the videos revealed great unevenness in how teachers interpreted the reforms and showed little evidence that classroom practices actually reflected the goals of the reforms.
We concluded from our first study that teaching is a cultural activity: learned implicitly, hard to see from within the culture, and hard to change. We were struck by the homogeneity of teaching methods observed within each country and by the striking differences in methods we observed across Germany, Japan, and the United States. Even in the United States, a country with great diversity in language, ethnicity, and economic conditions and an education system controlled by local governing boards, the nationwide variation in 8th grade mathematics teaching was much smaller than we had expected.
The TIMSS 1999 video study expanded on the first study. In addition to the United States, we included Australia, the Czech Republic, Hong Kong,1
Japan, the Netherlands, and Switzerland. Each of these countries performed significantly higher than the United States did on the TIMSS 1995 mathematics achievement test for 8th grade.
The design of the 1999 video study was simple. We selected a random sample of 100 8th grade mathematics classrooms from each country and videotaped them at some point during the school year. We digitized, transcribed, and translated the tapes into English, after which an international team of researchers analyzed them. Coding and analysis focused on the organization of lessons, the mathematical content of lessons, and the ways in which the class worked on the content as the lessons unfolded. Here are some of the most interesting findings.
In both the 1995 study and the 1999 study, teaching methods in Japan differed markedly from what we observed in all of the other countries. Japanese students, for example, spent an average of 15 minutes working on each mathematics problem during the lesson, in part because students often were asked to develop their own solution procedures for problems that they had not seen before.
Because Japan was the only high-achieving country in the first video study (as indicated by students' performance on the testing component of TIMSS), many researchers assumed that the United States would need to copy Japanese methods to produce the levels of learning displayed by Japanese students. The 1999 study, however, makes it clear that despite the well-crafted nature of the Japanese lessons, high achievement does not necessitate a Japanese style of teaching. Other countries posted high scores with lessons that looked decidedly un-Japanese. For example, in contrast to the relatively long time spent on each problem in Japan, every other country in the study spent only up to five minutes on the average problem.
The videotapes from each country reveal a unique combination of features. Many teaching methods that are hotly debated in the United States vary among the six higher-achieving countries. For example, the Netherlands uses calculators and real-world problem scenarios quite frequently. Japan does neither. Yet both countries have high levels of student achievement.
As another example, consider the debate over what kinds of problems students should work on during the mathematics lesson: basic computational skills and procedures (using procedures problems) or rich mathematical problems that focus on concepts and connections among mathematical ideas (making connections problems). Figure 1 shows the percentage of each kind of problem observed in six of the seven countries.
Japan is an outlier; 54 percent of the problems observed in the country's classrooms were making connections problems. But note that Hong Kong, one of the highest-achieving countries in the study, is at the opposite end of the continuum, with only 13 percent of problems coded as making connections. Classrooms in all of the countries spend time both on problems that call for using procedures and on those that call for working on concepts or making connections. The percentage of problems presented in each category, however, does not appear to predict students' performance on achievement tests.
What, then, do the higher-achieving countries have in common? The answer does not lie in the organization of classrooms, the kinds of technologies used, or even the types of problems presented to students, but in the way in which teachers and students work on problems as the lesson unfolds.
In the 1999 video study, we coded each problem twice: once to characterize the type of problem and the second time to describe how the problem was implemented in the classroom. The teacher could implement a making connections problem as a making connections problem, or the teacher could transform it into another type of problem—most commonly, a using procedures problem. For example, a teacher might transform a making connections problem designed to have students figure out a method for calculating the area of various types of triangles into a using procedures problem by giving students, at the outset, the formula (1/2 Base × Height) and telling students to simply plug in the relevant values.
Figure 2 shows how the teachers in the study actually implemented the making connections problems in the classroom: the percentage implemented as making connections and the percentage implemented as using procedures. Unlike Figure 1, this analysis reveals a pattern in which the highest-achieving countries resemble one another. Hong Kong and Japan, the two countries that differed most in the percentage of making connections problems presented, show a new similarity. In both countries, the majority of making connections problems are implemented as making connections problems; a much smaller percentage are transformed into lower-level using procedures problems. Here is the most striking finding of all: In the United States, teachers implemented none of the making connections problems in the way in which they were intended. Instead, the U.S. teachers turned most of the problems into procedural exercises or just supplied students with the answers to the problems.
The debates over mathematics education in the United States often pit two views against each other. One group believes that U.S. classrooms do not focus enough on concepts and understanding. The other group believes that U.S. classrooms overemphasize concepts at the expense of basic skills, thus holding back student achievement (Loveless, 2003).
Our research indicates that the lower achievement of U.S. students cannot be explained by an overemphasis on concepts and understanding. In fact, U.S. 8th graders spend most of their time in mathematics classrooms practicing procedures. They rarely spend time engaged in the serious study of mathematical concepts.
On the basis of this brief tour of the TIMSS videotape studies, three broad ideas can inform our efforts to improve classroom teaching of mathematics in the United States.
Most current efforts to improve the quality of teaching focus on the teacher: how the profession can recruit more qualified teachers and how we can remedy deficiencies in the knowledge of current teachers. The focus on teachers has some merit, of course, but we believe that a focus on the improvement of teaching—the methods that teachers use in the classroom—will yield greater returns.
The TIMSS video studies reveal that teaching is cultural; most teachers within a culture use similar methods. Indeed, within our study, teachers with strong mathematical knowledge showed the same cultural patterns of teaching as teachers with weaker knowledge. We must find a way to improve the standard operating procedures in U.S. mathematics classrooms—to make incremental and continuous improvements in the quality of the instruction that most students experience.
A focus on teaching must avoid the temptation to consider only the superficial aspects of teaching: the organization, tools, curriculum content, and textbooks. The cultural activity of teaching—the ways in which the teacher and students interact about the subject—can be more powerful than the curriculum materials that teachers use. As Figure 2 shows, even when the curriculum includes potentially rich problems, U.S. teachers use their traditional cultural teaching routines to transform the problems and reduce their instructional potential. We must find a way to change not just individual teachers, but the culture of teaching itself.
We can only change teaching by using methods known to change culture. Primary among these methods is the analysis of practice, which brings cultural routines to awareness so that teachers can consciously evaluate and improve them. A recent study by Hill and Ball (in press) of a large-scale professional development program found that analysis of classroom practice was one of three factors predicting growth of teachers' content knowledge.
Analysis of classroom practice plays several important roles. It gives teachers the opportunity to analyze how teaching affects learning and to examine closely those cases in which learning does not occur. It also gives teachers the skills they need to integrate new ideas into their own practice. For example, by analyzing videotaped examples of other teachers implementing making connections problems, teachers can identify the techniques used to implement such problems, as well as the way in which teachers embed these techniques within the flow of a lesson.
Attempts to implement reform without analysis of practice are not likely to succeed.
Finally, educators must find a way to inject new knowledge into the system of improvement and to share that knowledge with future generations of teachers (Hiebert, Gallimore, & Stigler, 2002). As John Dewey pointed out long ago, one of the saddest things about U.S. education is that the wisdom of our most successful teachers is lost to the profession when they retire.
What kind of knowledge do teachers need? They need theories, empirical research, and alternative images of what implementation looks like. U.S. teachers who want to improve their implementation of making connections problems, for example, will run up against a formidable challenge: They might never have seen what it looks like to implement these problems effectively.
Teachers need access to examples, such as those collected in the TIMSS video studies. They need to decide how they can integrate these examples into their own practice. They need to analyze what happens when they try something new in their own teaching: Does it help students achieve the learning goals? Finally, they need to record what they are learning and share that knowledge with their colleagues.
Teachers have a central role to play in building a useful knowledge base for the profession. Enabling teachers to learn about teaching practices in other countries and to reflect on the implications of those practices holds great promise for improving the mathematics instruction provided to all students.
Elmore, R. F. (2000). Building a new structure for school leadership. Washington, DC: Albert Shanker Institute.
Hiebert, J., Gallimore, R., Garnier, H., Givvin, K. B., Hollingsworth, H., Jacobs, J., Chui, A. M. Y., Wearne, D., Smith, M., Kersting, N., Manaster, A., Tseng, E., Etterbeek, W., Manaster, C., Gonzales, P., & Stigler, J. W. (2003). Teaching mathematics in seven countries: Results from the TIMSS 1999 video study (NCES 2003-013). Washington, DC: U.S. Department of Education.
Hiebert, J., Gallimore, R., & Stigler, J. W. (2002). A knowledge base for the teaching profession. Educational Researcher, 31(5), 3–15.
Hill, H., & Ball, D. (in press). Learning mathematics for teaching: Results from California's Mathematics Professional Development Institutes. Journal for Research in Mathematics Education.
Loveless, T. (2003, February). Trends in math achievement: The importance of basic skills. Presentation at the Secretary's Summit on Mathematics, Washington, DC. Available: www.ed.gov/inits/mathscience/loveless.html
Stigler, J. W., & Hiebert, J. (1999). The teaching gap. New York: Free Press.
This article refers to the Hong Kong Special Administrative Region of China as a country, along with the six other participants in the TIMSS 1999 video study, for the sake of consistency.
This article refers to the Hong Kong Special Administrative Region of China as a country, along with the six other participants in the TIMSS 1999 video study, for the sake of consistency.
Author's note: The TIMSS 1999 video study was funded by the National Center for Education Statistics and the Office of Educational Research and Improvement of the U.S. Department of Education, as well as the National Science Foundation. It was conducted as a component of the Trends in International Mathematics and Science Study (TIMSS), under the auspices of the International Association for the Evaluation of Educational Achievement (IEA). Each participating country provided the services of a research coordinator who guided the sampling and recruiting of participating teachers. In addition, Australia and Switzerland contributed financial support for data collection and processing of their respective samples of lessons.For complete details of the TIMSS 1999 video study, see Hiebert et al. (2003). This report and a four-CD set with 28 complete mathematics lessons for public release, four from each country, are available at www.lessonlab.com.Ronald Gallimore, codirector of the TIMSS 1999 video study, contributed to the ideas presented in this article. The views expressed in this article are the authors' and do not necessarily reflect those of the International Association for the Evaluation of Educational Achievement, the funding agencies, or any of the individuals who contributed to the studies.
James W. Stigler is a professor in the Department of Psychology, University of California, Los Angeles, and CEO of LessonLab, Inc.; firstname.lastname@example.org. James Hiebert is Rodney J. Barkley Professor in the School of Education, University of Delaware, Newark, Delaware; email@example.com.
Copyright © 2004 by Association for Supervision and Curriculum Development
Subscribe to ASCD Express, our twice-monthly e-mail newsletter, to have practical, actionable strategies and information delivered to your e-mail inbox twice a month.
ASCD respects intellectual property rights and adheres to the laws governing them. Learn more about our permissions policy and submit your request online.