Conference Countdown
Orlando, Fla.
October 31 - November 2, 2014
  • membership
  • my account
  • help

    We are here to help!

    1703 North Beauregard Street
    Alexandria, VA 22311-1714
    Tel: 1-800-933-ASCD (2723)
    Fax: 703-575-5400

    8:00 a.m. to 6:00 p.m. Eastern time, Monday through Friday

    Local to the D.C. area, 703-578-9600, press 2

    Toll-free from U.S. and Canada, 1-800-933-ASCD (2723), press 2

    All other countries (International Access Code) + 1-703-578-9600, press 2

  • Log In
  • Forgot Password?

 

Share
2014 ASCD Conference on Educational Leadership

2014 ASCD Conference on Educational Leadership

October 31–November 2, 2014, Orlando, Fla.

Learn the secrets to great leadership practices, and get immediate and practical solutions that address your needs.

 

Permissions

ASCD respects intellectual property rights and adheres to the laws governing them. Learn more about our permissions policy and submit your request online.

Policies and Requests

Translations Rights

Books in Translation

Premium Member Book (Aug 2003)

Collaborative Analysis of Student Work

by Georgea M. Langer, Amy Bernstein Colton and Loretta S. Goff

Table of Contents

Chapter 1. The Benefits of Collaborative Analysis of Student Learning

The Collaborative Analysis of Student Learning (CASL) system is designed to help teachers analyze student work to improve instructional decisions and, thus, students' learning. Student work is defined as any data or evidence teachers collect that reveals information about student learning (e.g., standardized test data, classroom assessments, writing samples, projects, oral reports, videotapes, pictures, and student observation data). As part of the system, teachers join a study group to interpret and document students' progress toward local learning standards and reflect upon how students learn as well as upon their own professional growth.

When the analysis focuses on the same students over an extended period of time, teachers make discoveries about how students construct meaning of key concepts and skills. As a result of the insights and skills gained through this system, teachers become much more purposeful about selecting instructional and curriculum approaches, moving students ever closer to the appropriate learning outcomes.

Components of the CASL System

The system is characterized by these four components:

  1. A framework for reflective inquiry
  2. A culture for collaborative inquiry
  3. The CASL inquiry phases
  4. Facilitation, leadership, and support
See Figure 1.1 and the following information for an introduction to each component; full descriptions are in later chapters.


Figure 1.1. Components of the Collaborative Analysis of Student Learning System


The Framework for Reflective Inquiry

  • Research- and theory-based conceptual map
  • A vision to guide teacher growth
  • Used for problem solving and adaptations

A Culture for Collaborative Inquiry

  • Norms that help the group function in a collaborative way
  • Specific communication skills that
    establish and maintain a safe and trusting environment
    encourage group members to reexamine, clarify, and transform their thinking so that they can help students succeed

The CASL Inquiry Phases

  • Define a target learning area as the focus of inquiry
  • Analyze classroom assessments to identify focus students
  • Meet in study groups to analyze student work, experiment with new strategies, and write documentation
  • Find more information to understand students, content, and strategies
  • Assess and analyze whole-class performance on the target learning area
  • Reflect upon student and teacher learning
  • Celebrate successes and share portfolios

Facilitation, Leadership, and Support

  • Facilitator
  • Administrative support
  • Structures (e.g., time, incentives)


Framework for Reflective Inquiry

We developed this system to portray and nurture what we believe underlies excellence in teaching—the teacher's ability to analyze reflectively what has happened, why it happened, and what to do next. We created the Framework for Reflective Inquiry (see Figure 2.1, p. 29) to synthesize research on the reflective teacher's characteristics, professional knowledge base, and a cycle for constructing new understandings (Colton and Sparks-Langer, 1993). While formulating our framework, the National Board for Professional Teaching Standards was creating a separate but similar model of “accomplished teaching,” called the Five Propositions (NBPTS, 1994).

For 12 years, we have used our framework to guide the development of various programs for student teachers, beginning teachers, and experienced teachers. Springing from this framework and experience is the system presented in this book.

Our framework is a key decision-making tool for us (and, we hope, for you) because it provides a vision of the responsible and effective teacher. Indeed, we do not believe the system described in this book is the only way to bring teachers toward this vision. It is, however, quite powerful, as indicated by the research. The framework can help with decisions about adaptations of the CASL system (i.e., “Do the proposed modifications maintain the vision in the framework?”). It also serves as a diagnostic tool to inform work with individual teachers (i.e., “Where in the framework is the teacher strong or weak, and how can I move the teacher closer to the vision?”). Chapter 2 provides a detailed explanation of the framework and how it relates to the system described in this book.

A Culture for Collaborative Inquiry

The context in which we learn is critical to the effectiveness of any professional development effort (Little, 1999; McLaughlin & Talbert, 1993). When collegial interactions are supportive and nonjudgmental, teachers may risk sharing the work of their less successful students.

During the months of CASL inquiry, teachers develop and maintain both norms for collaboration (Garmston & Wellman, 1999) and positive and productive communication skills. The norms provide ground rules to guide the group behavior, and the communication skills help to establish and maintain a safe and trusting environment and encourage group members to reexamine, clarify, and transform their thinking so that they can help students succeed.

We have found the system easier to implement in schools or districts where the staff has some experience working collaboratively on curriculum and instructional issues. In organizations with less experience or difficulties with collaborating productively, more explicit work on developing these skills may be required. Chapter 3 provides specific information on the norms and communication skills to support teachers' analysis of student learning.

The CASL Inquiry Phases

CASL engages teachers in the long-term analysis of individual students' learning. To begin the process, participants attend two or three workshops to (1) define a target learning area (e.g., examine test data, student assessments, standards, and test items to determine the focus of inquiry), (2) gather and analyze classroom assessment information, and (3) select two focus students for study (these students should exhibit signs of different struggles within the learning area).

Over the next three to five months, study groups (three to six teachers) meet every few weeks to analyze work samples from each teacher's focus students. A teacher begins by describing a student, the instruction that preceded the work, and the desired learning outcome (e.g., concept or skill). From there, the student work analysis cycle begins as the study group teachers make observations about the work, using descriptive, not evaluative, words. Then the group asks questions to prompt an analysis of the learning demonstrated in the work (e.g., “What does this work tell you about the student's understanding of the key skill or concept?” or “What do you know about this student that might explain why she performed in this way?” or “How effective do you think this teaching strategy was with her?”).

During this phase, the study group members refrain from giving advice or offering solutions. Indeed, one purpose of this work analysis is to formulate questions that might need further investigation. For example, the teacher might decide that it would be useful to hear the student “think aloud about how she did the work” before deciding what she needs. Sometimes teachers ask parents or other professionals about the student's interests and motivations. Teachers may also need to clarify how the specific understanding or skill relates to the school or district curriculum.

Finally, the group discusses a plan—teaching strategies that might help the student move to the next level of understanding or skill. Each of these potential actions is examined to see if it might help the student improve (at this point research literature and other experts are often consulted).

After the first student's work is analyzed, the same process is repeated for the next focus student. During the following weeks, the teachers implement the planned actions and collect the next sample of student work for another round of group observation, analysis, planning, and action. Over time, the group members discover what helps students learn and how to document their progress. Occasionally the study group finds it necessary to suspend the focus student work analysis to learn more about learning and teaching in the target area. This may involve consulting research, theory, and experts. After several months of meeting in study groups, teachers assess the entire class to determine progress on the target learning area.

Throughout the CASL inquiry phases, it's important to document the process. During the study group meetings, teachers take notes about how each focus student is progressing, why learning is or is not taking place, the short-term learning goal, the strategies to try, and which student work samples to bring to the next study group meeting. They also identify questions for further inquiry and how they will address them. In the final reflections, teachers include insights about student learning and their own teaching. This written documentation is compiled in the CASL portfolio as follows: (1) description of the target learning area, (2) analysis of the initial classroom assessment information, (3) reasons for selecting each focus student, (4) description of each focus student with written analyses of the work samples, (5) analysis of the whole-class final assessment, and (6) reflections on student and teacher learning.

This written dialogue with oneself is a powerful means to reframe problems, create new understandings of the nature of student learning, communicate information about students, and document teacher growth. At the project celebration, participants share the portfolios with the other teachers and leaders. Chapters 4 and 5 provide details about each phase of the inquiry process.

Facilitation, Leadership, and Support

Few innovations can survive without organizational support. The facilitator is one key piece of this system. An individual with effective communication skills should lead the workshops and study groups. The facilitator ensures that the norms are followed, that everyone develops the necessary communication and analytical skills, and that participants stay focused on student learning. The facilitator also encourages participants to reflect upon their own professional development and to seek outside information that may inform the group's work. Chapter 6 provides guidelines for facilitators.

Support from the school principal and other leaders also is critical to the success of the system. Principals need to “lead for professional learning, for organizational health, and for long-range and deep improvements” (Lieberman & Miller, 1999, p. 40). They need to read the school culture, analyze it, and decide how to prepare and support teachers for the analysis of student learning. Principals and leaders can do this by providing readiness activities, time, resources, encouragement, incentives, and support. Principals can also be responsive to CASL teachers' requests for specific workshops and other professional growth activities. Chapter 7 expands on these ideas.

What Are the Benefits of CASL?

We recognize the investment of time and money any new professional development project represents. You are probably wondering what evidence we have that this process is worth the effort. Our studies of more than 150 elementary and secondary teachers in 15 high-need schools from three different districts indicate that CASL is a powerful professional development system that positively affects teachers' thinking and practice, along with student learning.

Figure 1.2 (p. 18) presents the major studies we have conducted, listing the author, location, sample, and data sources. The richest data have come from the teachers' portfolios: evidence of whole-class and individual student learning plus the teachers' reflections on what has been learned about instruction, assessment, and student learning. A summary of the benefits is presented in Figure 1.3 (p. 19).


Figure 1.2. CASL Research and Evaluation Studies


Study

Sample

Topic

Duration

Data

L. Goff (Goff, 1999)

49 Mississippi elementary and secondary teachers

Writing across the curriculum

2 years, every 2–4 weeks

Interviews and portfolios

G. Langer (unpublished evaluation studies, 1999, 2001)

40 Michigan elementary and secondary teachers

Various learning areas in need of improvement

1 semester, every 2 weeks

Surveys and portfolios

A. Colton & G. Langer (unpublished evaluation study, 2001)

15 Michigan elementary teachers

Reading and writing

1 year, every 2 weeks

Surveys and portfolios

G. Langer & A. Colton (unpublished evaluation study, 2001)

20 Michigan middle school teachers

Math and science problem solving

4 months, every 2 weeks

Questionnaires


Figure 1.3. Outcomes and Benefits of CASL

Benefits to Students

The most important benefit of collaboratively analyzing student learning is that at-risk students learn more. Of the 55 students studied by Goff (1999), 90 percent showed improved learning in the work samples (Goff, Colton, & Langer, 2000). More recent CASL portfolios showed significant increases in the entire class's learning of elementary-level reading and writing skills (80 percent or more scored “proficient” in the Target Learning Area by the end of the year). As one 1st grade teacher said, “I have done assessment in the past, but have never taken the time to pinpoint my students' results on a chart to discover areas of strengths and weaknesses. Analyzing student work and trying different strategies has made all the difference.”

Benefits to Teachers

Commitment to and confidence in ability to promote student learning. One of our most consistent findings is that CASL teachers report and demonstrate a heightened commitment to improving student success, and they feel more confident in their ability to support student learning. As one middle school teacher stated,

Before CASL, I felt helpless. I would think, “The students aren't doing well . . . are they studying?” I didn't have the training to analyze their work. I would justify the lack of learning by thinking “Well, they just don't have the background” or “They missed something in an earlier class.” Now I ask myself, “Is there something I could do or say or some type of instruction that would get the lightbulb to come on?”
After participating in the system, teachers were more willing to entertain multiple factors that may be affecting their students' learning. They were less likely to write off some factors as “out of my control” and would try to overcome the many barriers that interfere with learning. For example, in the early sessions, one group of teachers made many hopeless references to poor children from a particular part of town. As they inquired into their learning and experimented with different strategies, the teachers found that they could have a positive effect on those students' learning, despite the economic challenges.

Analytical and reflective inquiry skills. Many teachers have reported growth in the analytical and reflective inquiry skills required to explore the link between their teaching and students' learning. The system changed their way of thinking by helping them discover, define, and analyze their students' problems. Then the teachers could identify teaching practices to correct the misunderstandings or fill in the gaps. As one teacher stated, “We become frustrated because we can't get students to produce what we want. This process gave me a better insight into why they couldn't do what I wanted.”

It is encouraging to see this type of analytical thinking about student performance become an automatic “self-questioning script.” At first, teachers followed the analysis process in a step-by-step manner to complete the portfolio. In time, however, the process became second nature. “It is automatic; I do it without thinking. The change is embedded. Before, I had to force it. I used the question sheets. Now I do it automatically.”

Professional knowledge. We've also seen teachers gain valuable professional knowledge about their curriculum, students, methods, assessment, and contextual factors. For example, a high school science teacher discovered that her expectations for writing assignments weren't clear because she wasn't sure how to teach writing. After seeking help from an English teacher in the study group, she was able to clarify outcomes and use new instructional strategies. She noticed (and documented) marked improvement in her students' writing.

Alignment among standards, instruction, and assessments. A large part of school reform involves the alignment of curriculum guidelines with assessments and instruction. As a result of their experience with CASL, many teachers realized that they could improve in this area. For example, one teacher noted, “If they were busy, then I hoped they learned something. But now I know.” This teacher explained that she had dutifully followed the teacher's edition of the textbook, but now she uses her assessment of student understanding of the standards to plan her instruction. Another elementary teacher stated, “This portfolio experience has changed my understanding of my subject matter to the extent that it has made me think more about this gap between teachers' expectations and students' actual successes (or lack thereof).”

Collaborative sharing of expertise (less isolation). We frequently receive comments about the opportunity to problem solve with other teachers. As one teacher explained, “We talked about different things that we could do. It helped to be able to work with someone instead of just trying to do it on your own.” As part of the portfolio reflections, one teacher stated, “I have learned a great deal from other teachers during this process.” We have also seen wonderful mentoring of new teachers occur in the study groups, for example, sharing of rubrics and assessment ideas. One first-year teacher stated, “It taught me to re-look and talk to everybody and get help.”

Awareness and self-assessment. During CASL sessions, many teachers come to surprising realizations about their professional practice. Typically these are private, internal “aha's.” For example, they may realize that they don't have a thorough understanding of a crucial concept in math or that a particular assessment does not really get at the target learning area or standards. And some teachers awaken to the uncomfortable fact that they have prematurely given up on a student. CASL is designed to provoke such insights in an environment of respect and commitment to student success. It is a safe place to admit a professional or personal shortcoming.

As one teacher remarked, “It made you think of what you were doing as a teacher. It made me look at my teaching styles. It made me really think about what I was doing and why I was doing it. I understood myself more.” A common statement found in the final reflections section of the portfolio is, “I now know I need more inservice training on [e.g., teaching of reading for comprehension].”

NBPTS certification. The first group of 49 CASL teachers included 12 who applied for National Board for Professional Teaching Standards (NBPTS) certification. All 12 went through this rigorous teacher assessment process and received the certification. They report that the CASL project helped them immensely, especially in writing about their analyses of student learning and the reflections on their practice.

This remark summarizes many of the ways CASL benefits teachers:

I think about teaching differently. I spend more time talking with students and listening to what they are saying. I ask, “Why did you do it this way?” The process made my strategies better and I feel like I have more ability in the classroom. I am a better teacher because I have a lot more confidence and I can figure out what their problems are.

Benefits to Parents and Organizations

We are happy to report that the system also benefits parents and organizations. The teachers' clarity about what they want their students to be able to do in the target learning area results in clearer communication with parents and other teachers. As one principal stated, “Parents love the fact that you can use work samples to talk to them about their children. ‘This is where your child started, and this is where he ended up.’” And a parent shared, “I've seen my child's portfolio from the beginning and . . . compared to now, he's achieved so much. . . . It's such a big difference.”

As teachers begin to discuss standards and assessments within grade levels, a higher degree of consistency can be reached. When such discussions occur across grade levels, teachers begin to delineate a logical sequence of skills from grade to grade. As teachers in one school compared the writing rubrics for each grade level, they found some inconsistencies in what was required and made changes to close the gaps. Such insights can have a positive effect on curriculum alignment and school improvement efforts.

The detailed information about students' learning is also useful in setting school improvement goals and for documenting student growth that results from curriculum and instructional interventions. One school found that its teachers did not thoroughly understand the specific nature of their students' reading problems. After using a reading assessment and charting the results they were able to find programs tailored to the needs of the students.

The student-learning data gathered early in the year can assist with school decisions about the allocation of resources. For example, as a result of analyzing each classroom's data on language-arts skills, staff at one school decided that some classrooms needed more paraprofessionals than others. Since all teachers were involved in interpreting the assessment data together, they agreed that this was the best way to help students succeed.

Professional-development planning is often easier after CASL because teachers are aware of where and how they need to grow. When teachers are masters of their own destiny, they are more likely to value what they learn and integrate it into their professional knowledge base. After studying students' writing in science, some teachers requested more training on the teaching of writing. In other groups, teachers might request training in multiple intelligences (MI) after hearing a colleague's ideas for using MI in the classroom. As one principal said, “It allows you to use your limited professional development monies in a more precise manner that meets teacher needs and student needs.”

One of the greatest benefits to the school community is the culture for collaborative inquiry developed through CASL groups, schools, and districts. As teachers and other leaders join together with the explicit purpose of helping students grow in a particular learning area, they develop a sense of group commitment, purpose, and collegiality. As stated by Sparks and Hirsch (1999), “Ultimately, the purpose of staff development on the local level is to change school culture and attitudes so that educators become better equipped to help all students reach high levels.”

Summary

CASL has four components: a guiding conceptual framework, a culture for collaborative inquiry, shared inquiry into students' learning, and supportive facilitation and leadership. Research indicates the positive effect of the system on student learning, teacher characteristics (e.g., commitment to and confidence in their ability to help students learn, professional knowledge and skills, and self-awareness), curriculum alignment, and organizations (e.g., collegiality, communication of student learning, and staff development). Such impressive outcomes are most likely to occur when the school culture supports teacher sharing of student work in a safe environment dedicated to learning about students' progress.