Chapter 3. Protocols for Examining Student Work

The protocols in this chapter are especially useful for examining student work, which usually leads to considering the professional practice that affected the work. The following protocols are featured:

• Tuning Protocol
• Rounds Protocol
• Vertical Slice Protocol
• Collaborative Assessment Conference Protocol

1 Tuning Protocol

Source: Joseph McDonald, David Allen, and others; the National School Reform Faculty (NSRF), which grants permission for its use.

Overview: This is the classic protocol upon which most of the others are based. It is also the most frequently used protocol for examining student work. The Tuning Protocol features time for the presenter to talk while participants are silent, and time for the participants to talk while the presenter is silent. It provides three levels of depth: presentation, participant discussion, and presenter reflection, finalized by a general debriefing that can extend the conversation.

Other Uses: This protocol can be used for examining teacher/educator practice; classroom, school, district, and other policies and practice; and plans, proposals, or ideas that are fairly well thought out and represented in written form.

Number of Participants: 8–10 participants, a presenter, and a facilitator

Time Required: Typically 1 hour; can range from 30 minutes to 2 hours

Steps (suggested times based on a 60-minute session):

Step 1: Introduction (first time only, 5 minutes)

• If participants don't usually work together, they briefly introduce themselves.
• Facilitator briefly introduces information about and guidelines for protocols, and establishes time limits for steps.
• Participants explore the assumptions that are important to making protocols work.

Step 2: Presentation (15 minutes)

• Presenter sets the context, describing the teaching/learning situation, while participants remain quiet and take notes.
• Presenter shares materials related to the teaching/learning situation described, including student work. The presenter should be prepared to use part of the presentation time to let participants examine what is being presented.
• Presenter poses one or two key questions about the teaching/learning situation.

Step 3: Clarifying Questions (5 minutes)

• Participants ask nonevaluative questions about the presentation (e.g., "What happened before X? What did you do next? What did Y say?").
• Facilitator guards against questions that approach evaluation (e.g., "Why didn't you try X?"). Participants who ask evaluative questions may be invited to rephrase the questions as clarifying, or to save the questions for the participant discussion step.
• It is entirely possible that the group will not get all its questions answered—there is never enough time!—but participants will have enough information at this stage for the protocol to be productive.

Step 4: Individual Writing (5 minutes). Both the presenter and the participants write about the presentation, addressing the key question(s). This step helps each participant focus and have something to say during the participant discussion.

Step 5: Participant Discussion (15 minutes)

• Participants discuss issues raised during the presentation among themselves, striving to deepen their understanding of the situation, and seeking answers to the question(s) posed by the presenter.
• The presenter is silent, taking notes on what the participants say. The presenter should avoid eye contact with the participants, even turning away from them, so that they focus on what is to be tuned rather than on the presenter.
• Participants should strive for a balance of warm and cool feedback unless instructed differently by the presenter.
• Participants should strive for substantive discourse. They should not engage in a round-robin discussion, but rather focus on and develop one idea at a time.
• The facilitator should keep an eye on the individual airtime of participants and ensure that their focus is on the work being discussed rather than on the presenter and that their comments remain true to the assumptions about protocols.
• Participants "own" the situation discussed during this step; it is theirs to improve, with the presenter listening in silently and taking notes, perhaps while turned away from the group to avoid eye contact.

Step 6: Presenter Reflection (15 minutes)

• The presenter reflects aloud on the participants' discussion, using the issues the participants raised to deepen understanding and reflecting on possible answers to questions posed. The presenter can also suggest future actions, questions, dilemmas, and so forth, and may correct any misunderstandings.
• Participants silently take notes on the presenter reflection.

Step 7: Debriefing (5 minutes)

• The presenter discusses how well the protocol worked and thanks the participants for their work.
• Participants discuss how well they think the protocol worked and thank the presenter for bringing the work to them to be tuned.
• The presenter and participants engage in more general discussion of both the situation examined and the protocol process itself.

Note: Ultimately, the person/group in charge of a particular step in the protocol is in charge of moving the process along, as well. When that person/group has no more to say, then that person/group can announce that it's all right to move to the next step.

Critical Elements:

• A balance of warm and cool feedback (unless the presenter has declared a need for more of one than the other)
• Attention to the presenter's key questions
• Thoughtful, provocative, and substantive discourse

Tips for the Facilitator: It might be helpful for participants to address the protocol process midway through the participant discussion. At that point, the facilitator may ask participants to assess how they are doing on the following:

• Balancing warm and cool feedback
• Addressing the presenter's key questions
• Ensuring that everyone has equal airtime

2 Rounds Protocol

Source: This protocol is a variation on The Descriptive Review of a Child by Pat Carini at the Prospect Center in Bennington, Vermont, for reflecting on students and their work, as described by Kelly (1996). This protocol is also based on ideas from Marilyn Wentworth and others at The Fulton Academy of Geographic and Life Sciences and Fort Pitt Elementary School in Pittsburgh, Pennsylvania; Vanessa Turpin, Trish Rygalski, and Jerome Morris of the Summer 1997 CFG Coaches Training Workshop; Steve Hoffman of the Alternative Community School in Ithaca, New York; and Steve Strull of DuSable High School in Chicago.

Overview: This protocol is very similar to the Tuning Protocol, except for the participant discussion, which consists of three rounds. During the first round, participants simply describe what they've seen in the student work examined. During the second round, participants reach some conclusions or generalizations about what they have described. During the third and final round, participants base a set of recommendations (both cool and warm in nature) on what they have concluded and generalized from the second round. Note that the descriptions in the first round of discussion should be objective. It is often hard for educators to describe rather than evaluate a work.

Other Uses: This protocol can also be used to understand a student through his or her work; in fact, this was the purpose of the Descriptive Review of a Child, on which this protocol is based.

Number of Participants: 8–10 participants, a presenter, and a facilitator. This protocol can also be done with multiple concurrent groups, each with its own presenter and facilitator, or with one presenter and multiple concurrent groups.

Time Required: Typically 45 minutes to 1 hour

Steps (suggested times based on a 60-minute session):

Step 1: Introduction (first time only, 5 minutes)

• If participants don't usually work together, they briefly introduce themselves.
• Facilitator briefly introduces information about and guidelines for protocols, and establishes time limits for steps.
• Participants explore the assumptions that are important to making protocols work (see p. 13).

Step 2: Presentation (10 minutes)

• Presenter sets the context, describing the teaching/learning situation, while participants remain quiet and take notes.
• Presenter shares materials related to the teaching/learning situation described, including student work. When student work is being presented, presenter should allow participants time to examine the work.
• Presenter poses one or two key questions about the teaching/learning situation.

Step 3: Clarifying Questions (5 minutes)

• Participants ask nonevaluative questions about the presentation (e.g., "What happened before X? What did you do next? What did Y say?").
• Facilitator guards against questions that approach evaluation (e.g., "Why didn't you try X?"). Participants who ask evaluative questions may be invited to rephrase the questions as clarifying, or to save the questions for the participant discussion step.
• It is entirely possible that the group will not get all its questions answered—there is never enough time!—but participants should have enough information at this stage for the protocol to be productive.

Step 4: Individual Writing (5 minutes). Both the presenter and the participants write about the presentation, addressing the key question(s). This step helps each participant focus and have something to say during the participant discussion.

Step 5: Participant Discussion (15 minutes). The participants move through the following rounds. If possible, a recorder writes what participants say on chart paper. The presenter remains silent and takes notes throughout.

• Round 1: Description (5 minutes). In round-robin style, participants describe what they do (or do not) see in the work (e.g., "The student indents for new paragraphs"). Participants can pass if they have nothing to add.
• Round 2: Generalization (5 minutes). In round-robin style, participants make generalizations about what they do (or do not) see in the work, based on the descriptions from the first round (e.g., "The student uses indention erratically"). Participants can pass if they have nothing to add.
• Round 3: Recommendations (5 minutes). In round-robin style, participants make recommendations based on the descriptions and generalizations from the previous two rounds (e.g., "The student needs to learn the rules for forming paragraphs"). Participants can pass if they have nothing to add. The group should make every effort to offer warm and cool recommendations; warm recommendations indicate what works and should be continued, and cool recommendations indicate what needs improvement. The group should also make every effort to address the presenter's key questions about the work.

Step 6: Presenter Reflection (15 minutes)

• The presenter reflects aloud on the participants' discussion, using the issues the participants raised to deepen understanding and reflecting on possible answers to questions posed. The presenter can also suggest future actions, questions, dilemmas, and so forth, and may correct any misunderstandings.
• Participants silently take notes on the presenter reflection.

Step 7: Debriefing (5 minutes)

• The presenter discusses how well the protocol worked and thanks the participants for their work.
• Participants discuss how well they think the protocol worked and thank the presenter for bringing the work to them to be tuned.
• The presenter and participants engage in more general discussion of both the situation examined and the protocol process itself.
• The facilitator engages participants in discussion of the three rounds and why they are important.

Critical Elements:

• A balance of warm and cool feedback (unless the presenter has declared a need for more of one than the other)
• Attention to the presenter's key questions
• Thoughtful, provocative, and substantive discourse

During the participant discussion, the facilitator may also help the participants to stick to nonevaluative descriptions in the first round, and base both generalizations and recommendations on these descriptions. If participants make generalizations or recommendations that are not based on previously made descriptions, the facilitator should encourage them to come up with relevant descriptions.

Tips for the Facilitator: One of the best activities the facilitator can engage participants in before starting this protocol is practice with description. For example, the facilitator could start by asking participants to describe the room they are in. If a participant says, "It's crowded," the facilitator could gently point out that this is a generalization. Participants should simply describe the size of the room, its furnishings, the number of people in it, and so forth before concluding that it's crowded. Similarly, if participants say that the room is cold, they should first establish the temperature; if they volunteer that the room is noisy, they should first describe all the noises in the room. As further practice, they might then try describing a piece of student work that is not being used for the protocol.

3 Vertical Slice Protocol

Source: In 1996, the Bush Educational Leadership Program at the University of Minnesota worked with the Prairieville, Minnesota, school district to create the Vertical Slice (also known as the Minnesota Slice), which it used to capture student data for use in analyzing the purposes of education. Variations on this process include the Albuquerque Slice, created by school coaches from the National School Reform Faculty of the Annenberg Institute for the 1996 Fall Forum of the Coalition of Essential Schools; the Longfellow Slice; the Columbus Family Academy Slice; the "Day in the Life" Slice; the South Bend Slice; and the Hempstead Slice.

Overview: This protocol focuses on an examination of all the student work produced during a narrow time period by a sample of students in a particular school or district.

Other Uses: The Vertical Slice Protocol can be used for a variety of purposes, depending entirely on what the group engaged in the slice wants to know. It can be used to help educators understand students' perceptions of the school, some aspect of a problem or issue, how rigorous classroom work is, how interdisciplinary curriculum functions, and so forth.

Number of Participants: Any number of people can participate in this protocol, but groups of 20 or more might work better if broken into groups of 10 or so. If groups are broken up, it is important for them to consolidate their learning. The Vertical Slice can involve a single grade level or adjacent grade levels; subject area teachers; an entire school staff, including counselors and administrators; educators across schools, alongside district administrators; and parents and community members.

Time Required: Typically 1–2 hours for the planning meeting and 3–4 hours for analysis.

Steps:

Part 1—Planning Meeting: The planning meeting involves either the entire group that will be participating in analysis or a representative group. The steps for the planning meeting are as follows:

Step 1: Determining the Purpose of the Slice (up to 15 minutes)

Step 2: Determining a Guiding Question Related to the Purpose of the Slice (up to 15 minutes)

Step 3: Determining How Student Work Is to Be Obtained (up to 15 minutes). Each group will need to think of its own needs and design its own type of student work to collect. Here are some examples:

• One student's work throughout one day
• A sample of work from randomly chosen students in one grade on one day
• A sample of work from randomly chosen students across grades on one day
• A sample of work from one randomly selected student in the same grade in each of several schools
• Samples reflecting work from students at certain socioeconomic levels or levels of English fluency
• Samples of work from students enrolled in special education courses, AP courses, art classes, and so forth

Step 4: Identifying Other Aspects of the Work to Be Collected (up to 15 minutes). These aspects include the following:

• What the sample will consist of (e.g., work on paper, videos, artwork, photos, journals, audiotapes, student logs/reflections)
• Whether the context of the work will be examined in addition to the work itself (e.g., the assignment, the instruction leading up to the assignment, whether or not students worked together or individually)
• Whether the work will be anonymous or identified

Step 5: Deciding on the Duration of the Slice (up to 15 minutes). Although slices usually consist of a day's worth of work, consider collecting during a particular hour or a certain period of the day; alternatively, consider examining work that has been collected over a longer period of time (a week, for example) and then randomly selected for the analysis. Be careful not to collect too much work.

Step 6: Attending to the Logistics of the Collection Process (up to 15 minutes)

• Answer these critical questions: Who will collect the work? If random selections are to be made, who will make them? How will parents be informed of the process? Do they need to give permission for the school to analyze their students' work, even if the students remain anonymous?
• Organize the collection. If selecting at random from the collection, do so at this point.
• Make copies of the work so that everyone has the same final collection.
• Establish a time and place for the analysis and distribute this information to those involved.
• Gather refreshments and tools for analysis (paper and pencil, laptops, etc.).
• Decide on a facilitator.

Step 7: Determining How the Analysis Will Be Conducted (up to 15 minutes). Will participants scan all of the material and then focus on representative pieces? Will the dialogue be Socratic? Will groups be large or small?

Step 8: Determining Questions to Ask During the Protocol (up to 15 minutes). Here are some examples from the National School Reform Faculty ("Sample Sets of Questions for School/Grade Level Slice," n.d.):

• What evidence is there that students develop and apply essential knowledge and skills in challenging and meaningful ways?
• What evidence is there that … gaps exist within the curriculum?
• What evidence is there … of redundancy or unnecessary overlap within the curriculum schoolwide?
• What evidence is there that the student work builds on individual learning styles and skill levels of students and fosters student self-expression?
• What evidence is there that lessons encourage students to develop and apply problem-solving abilities?
• What essential skills and proficiencies in language arts, social studies, and mathematics are being applied or developed through the student work?
• What evidence is there that individual learning styles and skill levels are being incorporated into the lessons?
• What evidence is there … of thematic connections being made across the curriculum?
• What essential skills and proficiencies in language arts and mathematics are being applied or developed through the student work? (p. 1)

Part 2—Analysis (suggested times based on a 130-minute session):

Step 1: Preparation (up to 15 minutes). The facilitator establishes norms, facilitates introductions, and explains the process.

• If the group is large, the facilitator breaks the whole group into smaller groups for the purpose of analysis.
• The facilitator briefly describes the parameters and methodology of the protocol.
• The facilitator presents the guiding question for the discussion.

Step 2: Examining the Work (up to 50 minutes). Participants examine the work and take notes in silence. Small groups may examine different blocks of evidence in order to cover all the work presented.

Step 3: Discussion (up to 90 minutes)

• If the group is large, the facilitator leads one group in the discussion, using previously introduced norms, while the other groups silently take notes.
• Participants in each group share their thoughts about guiding questions and about any other questions that might have arisen during their examination; they also attempt to identify themes or trends.
• The process is repeated until each group has had a chance to lead the discussion. With each round, the discussion should become deeper as participants build on what they have heard.

Step 4: Framing Answers (up to 55 minutes). The whole group works together to frame some answers to the guiding question and to questions designed during the planning process.

Step 5: Debriefing (up to 15 minutes). The facilitator leads the whole group in debriefing. What have participants learned through this process, and why? What could be improved? The group identifies potential next steps for deepening the student work related to the guiding questions.

Critical Element: Guiding questions that are clear and focused. The questions under Step 8 of the first part of the process are clear and focused.

Tips for the Facilitator: This is a complex protocol and succeeds to the extent that it has been well prepared for. The facilitator needs help from individuals or groups, such as a representative design team, to plan the protocol so that its results are meaningful. The facilitator needs logistical support to make sure the process is smooth and efficient.

4 Collaborative Assessment Conference Protocol

Source: Steve Seidel, Director of Project Zero at Harvard, devised this protocol for a group of educators that gathered each weekend to look at student work. The protocol is described in Blythe, Allen, and Powell (1997) and McDonald, Mohr, Dichter, and McDonald (2003).

Overview: This protocol is unique because it does not feature information about context (assignment, classroom, student, etc.) at the beginning; instead, participants are encouraged to look at the work by itself. Steve Seidel describes the four main purposes of the protocol as follows:

The first is to enhance teachers' perceptions of all their students' work by honing the teachers' perceptual skills. A second is to encourage depth of perception by demonstrating all that can be seen in a single student's work. A third is to encourage a balance in perception—the habit of looking for strength as well as need. The assumption behind this purpose is that a teacher can address need only by building on strength. A fourth purpose is to encourage conversation among teachers about what the work shows and how they can act individually and collectively on what it shows in order to benefit their students. (McDonald et al., 2003, p. 77)

The National School Reform Faculty elaborates on the key ideas behind the protocol ("Collaborative Assessment Conference," n.d.):

First, students use school assignments, especially open-ended ones, to tackle important problems in which they are personally interested. Sometimes these problems are the same ones that the teacher has assigned them to work on, sometimes not.

Second, we can only begin to see and understand the serious work that students undertake if we suspend judgment long enough to look carefully and closely at what is actually in the work rather than what we hope to see in it.

Third, we need the perspective of others—especially those who are not intimate with our goals for our students—to help us to see aspects of the student and the work that would otherwise escape us, and we need others to help us generate ideas about how to use this information to shape our daily practice. (pp. 3–4)

Number of Participants: 5–15 participants, a presenter, and a facilitator

Time Required: 45–90 minutes

Steps (suggested times based on an 85-minute session):

Step 1: Introduction (5 minutes). The facilitator makes sure that everyone knows everyone else and explains the protocol, including the suggested time for each step.

Step 2: Sharing the Work (5 minutes). The presenter shares the student work with the participants but says nothing about the piece, the conditions under which it was produced, or the student.

Step 3: Examining the Work (10 minutes). Participants silently examine and take notes on the work.

Step 4: Describing the Work (10 minutes)

• The facilitator asks the group to provide nonevaluative descriptions of the work using questions such as "What do you see?" "What's there?" and "What's not there?"
• If a participant provides evaluative commentary, the facilitator asks him or her to rephrase the comment as a description of evidence upon which the opinion might be based.

Step 5: Raising Questions (10 minutes)

• The facilitator asks the group to consider questions that the work raises for them using questions such as "What came to your mind as you examined this work?" "What did you notice?" "What struck you?" and "What questions does this raise for you?"
• Participants respond with questions about the conditions under which the work was produced, the student, the context, or the work itself.
• The facilitator might record these questions on a piece of chart paper or ask someone to serve as recorder during this step. (The presenter remains silent during this step.)

Step 6: Speculation (10 minutes)

• The facilitator asks participants to speculate on what the student is working on, both personally and academically.
• Participants make suggestions about the problems or issues that the student might have focused on in creating the work.

Step 7: Presenter Reflection (10 minutes)

• The facilitator asks the presenter to address any questions or speak generally about the student whose work is being examined or the context of the work.
• The presenter may offer a perspective that relates to the work, including what he or she sees in it. However, the presenter does not need to answer any or all of the questions.
• Some presenters share what they found surprising about the participants' comments during steps 4–6.

Step 8: Implications of the Work (15 minutes)

• The facilitator invites everyone (including the presenter) to address the implications of the work and their analysis of it. A typical facilitator question at this point might be, "What have we learned by examining this work that can help us in our own teaching?"
• Participants can discuss their own teaching, how people learn, or how the student whose work was examined can be supported in the future.

Step 9: Debriefing (10 minutes). The facilitator invites the whole group to debrief the experience—both the content of the conference as well as the process—after thanking the presenter and offering him or her a chance to reflect.

Critical Elements: Participants may have difficulty sticking to nonevaluative descriptions of the work and may need help rephrasing their comments. Similarly, they may be unsure about what kinds of questions they can raise; almost any question about the work, its context, or the student is appropriate. Finally, it is extremely important for participants to see beyond the particular student and work being examined to general teaching and learning strategies.

Tips for the Facilitator: The facilitator plays a very active role in this protocol; steps 4–6 in particular require the facilitator to provide prompts and probably examples. The facilitator may need to have participants practice making nonevaluative descriptions before the process begins by surveying the room (see the tips for the facilitator under the Rounds Protocol) or analyzing a piece of writing or artwork.

The facilitator may also need to reassure participants that they can, indeed, do this protocol without knowing the context of the student work.

MacDonald and colleagues (2003) suggest that facilitators also need to "press participants to go deeply into the work, to raise more questions and make more speculations collectively than any one member imagined possible" (p. 79). Facilitators need to help participants surmount superficiality. For example, when asked what a student seems to be working on, a participant might say, "a math problem." The facilitator should prod the speaker to think about the learning the student is going through to do the math problem.

Although some approximate times are given above, the facilitator must have a keen sense of when a group is finished and ready to move on or needs to stay on a step and push thinking deeper.