|
December 2008/January 2009
| Volume 66 | Number 4
Data: Now What?
Driven Dumb by Data?
Marge Scherer
The Spectrum of Education Research
Jeffrey R. Henig
When education issues come to be understood in terms that neatly overlay the standard ideological alignments, they often become proxies for the war between Right and Left. This has been the case in the instance of charter schools, with proponents and critics mustering up researchers and findings to buttress their respective claims and, in general, resisting compromise. Although this politicization can be dispiriting, there are some encouraging signs that research in education is improving. The field is attracting better-trained researchers, better data on student achievement are available, and research designs are becoming more rigorous. Research as a collective enterprise—comprising multiple studies, attacking questions from differing angles, and exposed to the checks and balance of other studies in different settings and using differing methodologies—can increasingly enrich our knowledge base.
The New Stupid
Frederick M. Hess
A decade ago, many education leaders dismissed student achievement data and systematic research as having only limited utility when it came to improving schools or school systems. That was the "old stupid." But now the pendulum has swung the other way. In the "new stupid," data-based decision making and research-based practice can stand in for careful thought, serve as rationales for the same old fads, or be used to justify incoherent proposals. The "new stupid" uses data in half-baked ways, is characterized by trouble translating research, and gives short shrift to management data. Educators can steer clear of the new stupid by remembering the limitations of data, seeking out appropriate data, and applying judicious thinking.
Answering the Questions That Count
David Ronka, Mary Ann Lachat, Rachel Slaughter and Julie Meltzer
Despite the increased amounts of data available, many educators still feel ill prepared to analyze and use their data effectively. Schools need a framework for building data literacy. By organizing data use around a cycle of inquiry and grounding it in the core components of systemic data use—data quality, data capacity, and data culture—schools can promote positive change. Data quality means that data must be sufficiently disaggregated to address questions of concern; data capacity means establishing data teams, designating data coaches, and creating structured time in the school calendar for collaborative analysis; data culture means paying deliberate attention to issues of leadership, policy, accountability, shared beliefs, and collaboration.
Accountability with Roots
George Johnson and Susan Bonaiuto
If accountability doesn't grow out of local priorities, the authors write, it loses meaning. Externally mandated data (read: test scores) rather than community priorities then become the focus for education conversations. To avoid this scenario, Needham Public Schools in Massachusetts initiated a series of focus groups to collect feedback from community members about what kind of data they hoped to have to evaluate their schools' performance. The authors discuss what these groups revealed about their community's views on three questions: (1) What are the qualities of an excellent school or school system? (2) What core competencies should we expect from high school graduates? and (3) What evidence would indicate that our graduates had achieved these competencies? Focus groups revealed that citizens valued many student competencies other than academic achievement and considered scores from the state's standardized test unimportant. On the basis of these results, the district developed an annual performance report on Needham's schools that provides data measuring the competencies the community listed as important (such as students' engagement in learning and college attendance rates).
Measuring the Achievement Elephant
Paul E. Barton and Richard J. Coley
Barton examines some important aspects of various methods of measuring student achievement, pointing out that school leaders need to understand the meaning of test score data before they can use it wisely. For example, he points out that most accountability systems are based on the assumption that changes in the proportion of students who score at or about a predetermined cut-point tell the whole story of school improvement—but this measure provides no information about changes in the achievement of students who remain above or below the cut-point. Another assumption is that No Child Left Behind requires that achievement gaps be closed by 2014; actually, the law requires only that all defined subgroups reach the "proficient" level by that year; a state could well achieve this goal while maintaining exactly the same gaps in average scores that it had before.
School Climate Through Students' Eyes
Bill Preble and Larry Taylor
When school district leaders in Sullivan County, Tennessee, were placed under a court order to address issues of bullying and harassment in the district's schools, they saw it as an opportunity to do what was right and to ensure that every student was safe. They asked the authors and their colleagues to develop a plan to collect school climate data and use these data to guide school improvement. This article describes the student-led, collaborative action research process that Sullivan County implemented. The authors have implemented this process successfully to assess and improve school climate in hundreds of schools throughout the United States.
The Assessment Double Play
Roberta Buhle and Camille L. Z. Blachowicz
Some things that look easy are actually the result of hard work and thoughtful orchestration. Such is the case with moving from formative assessment to data analysis to effective instruction. The authors of this article, experienced literacy coaches, assert that it is naïve to believe that teachers will automatically use assessment data to inform instruction without the coaching and support they need to begin the process. The article describes two cases in which literacy coaches helped teachers build bridges between their professional knowledge and the assessment data they had obtained, resulting in improved instruction.
Researchers and Educators: Allies in Learning
Mark Dynarski
In contrast to the training that doctors receive, education training programs put little emphasis on training students to conduct research or interpret and use research findings. According to the What Works Clearinghouse, relatively few rigorous studies in education exist. As in the field of medicine, controlled trials can provide a wealth of information. Researchers and funders are now seeing the importance of district buy-in—of collaborating with district educators to answer not only their own questions, but also the questions the districts may have—to conducting long-term, rigorous studies. The Institute of Education Sciences is doing more to meet the needs of districts by funding studies that provide districts with useful information they might not otherwise be able to collect.
The Collaborative Advantage
Jennifer L. Steele and Kathryn Parker Boudett
Schools that explore data and take action collaboratively provide the most fertile soil in which a culture of improvement can take root and flourish. The authors describe how eight schools, working with the Harvard Graduate School of Education's Data Wise improvement process, have used teacher collaboration to understand student achievement data and develop realistic instructional improvements. They describe how school leaders can create the right conditions for effective collaboration by allocating time for the process, delegating data management to data coordinators or teams, and establishing norms of positive intentionality, transparency, and objectivity that foster trust.
Data Beyond High School
Elliot Washor, Karen Arnold and Charles Mojkowski
Most schools stop collecting data on students after high school graduation. But schools associated with the Big Picture Company have in place a system to collect data on the life trajectories of their graduates for 12 years after high school. The authors describe Big Picture schools' system of collecting data on graduates through their alumni manager databases. Schools administer surveys to all graduates right before graduation (probing how well they feel their high school experiences prepared them for college) and during their first year after graduation, and seek annual updates thereafter. In keeping with Big Picture's view that success encompasses more than academics, surveys track not only students' academic achievements, but also the quality of their relationships, their feelings about their transition to college, and similar questions. The authors discuss what tracking student outcomes revealed to their schools, particularly in terms of how low-income students need more support to make attending and completing college realistic.
Student-Driven Research
Makeba Jones and Susan Yonezawa
For the past two years, the Center for Research on Educational Equity, Assessment and Teaching Excellence at the University of California, San Diego, has designed and run Student-Created Research projects in eight racially diverse urban and low-income San Diego high schools. Students undertake research projects intended to guide school improvement efforts and share their findings with school faculty. In this article, Yonezawa and Jones describe how students at the School of International Studies surveyed students and teachers about how the school's structure and typical student workload affected students' attitudes. Such efforts enable students to learn valuable research skills while giving school faculty important information to drive their planning and professional development.
Measuring What Matters
Mike Schmoker
Throughout the last decade, Schmoker asserts, educators have come to see data as an indispensable tool for school improvement. However, the marriage between the data-driven instruction movement and NCLB has unintentionally made instruction less effective and relevant in many schools. To a large extent, Schmoker points out, data-driven decision making is in reality standardized test data-driven decision making, and this focus has lead to "test prep" style instruction that does nothing to promote such skills as critical thinking, problem solving, the ability to argue convincingly, literacy, and invention—skills that will be sorely needed in the 21st century. Schmoker describes one model—the New York Performance Standards Consortium—that uses data to drive instruction in service of such 21st century skills. All 28 high schools in the consortium have students work toward culminating academic projects that draw on higher order skills, and for which clear rubrics have been developed. Students' performance on these rubrics is the main data the schools focus on. Teachers continually analyze rubric performance and use that data to make changes in instruction designed to sharpen 21st century skills.
A Day's Worth of Data
Margery B. Ginsberg and Catherine Brown
Using Data, Changing Teaching
Barnett Berry, Carolann Wade and Paula Trantham
Most education researchers and policymakers agree that good teachers are essential to student success. What is less clear is what makes a good teacher. What working conditions are most likely to lead teachers to success? The National Center for Teaching Quality has been surveying teachers to find out what working conditions matter most to them. In this article, Berry, Wade, and Trantham share some of the survey findings and describe how one school has used working-conditions survey data in its improvement efforts. In a partnership with Peace College, Millbrook Elementary School in Raleigh, North Carolina, under Trantham's principalship, instituted professional learning communities and reduced duty assignments that took teachers away from teaching and learning. Teachers have responded positively to these changes. More data are needed, however, before researcher can determine whether these improvements in working conditions for teachers can improve student achievement.
Anchoring Down the Data
W. James Popham
Collaborative Inquiry
Jane L. David
Looking Deeper Into the Data
Douglas B. Reeves
Learning About—and From—Data
Amy M. Azzam
Data That Count
Thomas R. Hoerr
ASCD Community in Action
Bits from the Blog
Journal Staff
Why Teachers Must Be Data Experts
Jennifer Morrison
Morrison, a high school language arts teacher, discusses why classroom teachers need to view data as a resource to help investigate authentic questions arising from their practice, rather than an element pushed on them by administrators. Administrators and teacher leaders must help educators (1) realize that data includes more than end-of-year standardized test scores; (2) view collecting data as a way to investigate the many questions about students, teaching practices, and learning that arise for teachers; and (3) dialogue with each about what data reveal and how to build on those revelations.
A Tale of Two Schools
Jennifer D. Morrison and Margaret Rudt
Two middle schools in Montgomery County, Maryland, were struggling to make adequate yearly progress (AYP). At John T. Baker Middle School, a large percentage of students were achieving proficiency, but scores for certain subgroups of students were low. Col. E. Brooke Lee Middle School was working hard to help students who come to school with significant skill gaps, but they weren't making enough progress. Both schools developed a strategic monitoring and intervention program that required them to look closely at student data and tailor their interventions to student needs. Teams of educators met together regularly to discuss student progress and adjust plans as needed. By 2008, both schools had made AYP for two years in a row and had moved out of school improvement status. They attribute their success to the strategic monitoring and intervention plan.
Raising the Bar at Furness High
Robert E. Slavin, Gwen Carol Holmes and Cecelia Daniels
Furness High School, a Philadelphia school serving low-income students, had never made adequate yearly progress under NCLB and was beginning to think it never would. Classroom observations revealed a "devil's bargain" between unmotivated students and overwhelmed teachers: Teachers presented unchallenging instruction that left potential troublemakers free to socialize or sleep in exchange for those students not disrupting lessons. After working with the Raising the Bar program developed by the Center for Data-Driven Reform in Education, Furness saw student engagement and test scores rise. Making AYP is now no longer a problem. Furness not only analyzed data on lagging student achievement, but also brought students into the data analysis process and adopted cooperative learning methods that motivated students to change this achievement picture. Teachers were trained to apply cooperative learning methods. Students worked in teams to help one another master learning objectives for group rewards. The default instruction format at Furness switched from teachers presenting information to bored students to learners enthusiastically strategizing together to boost achievement.
The Challenge of Assessing School Climate
Jonathan Cohen, Terry Pickeral and Molly McCloskey
When students feel safe, connected, and engaged in school, they are more likely to learn well. Research has confirmed that positive school climates promote learning—yet this knowledge has not been translated into current accountability systems. No Child Left Behind accountability focuses on academic learning to the exclusion of the whole child. The authors of this article advocate using comprehensive school climate surveys to measure the extent to which students are healthy, safe, engaged, supported, and challenged in school. Comparing the responses of various stakeholder groups (students, parents, school staff, and community members) provides valuable information that can guide school improvement efforts.
Taking Data to Heart
Kim K. Metcalf
Educators can become more proficient in using data when they acknowledge—and counteract—some common human behaviors. The author shares five lessons learned: (1) We tend to be less critical of data that support what we already believe and more critical of data that do not; (2) O = T + E explains a lot about why we do what we do, such as why we often choose to focus on the error in a set of observations that we don't like and overlook the error in observations that we do like; (3) It's easier to focus on what we do rather than why we do it or how we'll measure success; (4) Data are good, invaluable tools; and (5) Sometimes doing what is most effective isn't worth what we have to give up.
EL Study Guide
Teresa Preston
Copyright © 2004 by Association for Supervision and Curriculum Development
|