Midway through his sophomore year at Brighton High School in Boston, Mass., Jerry Anderson was in the bottom of his class, at risk of failing the 10th grade state exam, and unlikely to graduate on time. So how was it that by the end of the academic year, Jerry earned a proficient score on his state exam and was back on track to graduate, while at the same time his school's overall scores went up by 23 percentage points from the prior year?
Richard DuFour (2004) recommends that schools ask the question "How will we respond when some students don't learn?" For Jerry and 37 other 10th graders like him, Brighton High administrators and teachers asked, "How will we respond when some students are at risk of failing?" They found their answer in the design and implementation of a targeted intervention that provided critical literacy instruction beyond the school day for students at risk of failing the state exam.
Why Intervene?
Brighton High School, one of 35 Boston public high schools, is a diverse, urban high school serving 1,251 students, including large percentages of low-income families, English language learners, and students with disabilities. At the time of the intervention, the school was structured in three small learning communities, each serving a cross-section of the school community.
For four consecutive years prior to the intervention, students had averaged a 75 percent pass rate on the 10th grade English Language Arts (ELA) Massachusetts Comprehensive Assessment System (MCAS), a standardized state assessment used as a graduation requirement. This passing rate put Brighton High among the top of the nonselective district high schools within Boston, and was a vast improvement over the school's low 25 percent pass rate a decade earlier. The school leaders attributed their success to their professional learning community of highly collaborative, talented, and reflective teachers and administrators.
Despite this accomplishment, Brighton High could not rise above the 75 percent pass rate on the ELA test. If the trend continued, it meant 25 percent of their current 10th graders would fail. Failing the ELA MCAS indicated that these students lacked the literacy skills to succeed in their classes. Research showed that low-income urban students with low MCAS scores have less of a chance of graduating from high school (Papay, Murnane, & Willett, 2008). The school district had set a goal for every student to be "on track" to graduate by the end of 10th grade; in order for these students to succeed, administrators and faculty at Brighton High knew that they had to intervene.
Understanding the Dilemma
Brighton High's ELA MCAS data from the previous year indicated that the 10th graders had failed because of poor scores on the multiple choice reading comprehension portion of the test. Realizing these students needed more explicit literacy instruction and confidence in their abilities, teachers collaborated in study groups to adapt their classroom instruction to help low-skilled readers; nevertheless, students like Jerry did not respond. Without a systematic approach to the problem, these students would fail the MCAS.
Brighton High assembled a team of administrators, the literacy coach, the ELA teacher leader, 10th grade teachers, and staff from the Boston Plan for Excellence (BPE), a nonprofit district partner, to create and administer a targeted intervention for specific students that was skill-based, narrow in scope, and based on the results of a common reading pre-assessment.
Plan, Do, Check, Act
Influenced by the literature on professional learning communities, the team set out to develop the intervention. For an intervention to achieve desired results, DuFour and others (2006) reference Edward Deming's four-step feedback loop: "Plan: Design the process to improve results; Do: Implement the plan and measure its performance; Check: Analyze, assess and report on the results; Act: Decide what changes must be made to improve the process, and adjust accordingly" (p. 152). This loop guided our actions.
Plan
Target Students: Who Needs an Intervention?
To identify which 10th grade students were at risk of failing the ELA MCAS, the team considered students' grade point averages, class grades, and a rigorous MCAS-style formative reading assessment designed by BPE. BPE aligned the results of the assessment with MCAS results from prior years and identified those students at risk of failing. Although the team wanted to include all at-risk students in the intervention, the MCAS was three months away, so the team focused on a target group who could most benefit from this particular intervention.
The team chose 45 students were chosen based on three criteria: attendance, performance, and equity.
Attendance: Each student's attendance record had to be at least 75 percent present for the school year thus far, to ensure sufficient exposure to the strategies taught in class and in the intervention.
Performance: Each student had to have scored "on the bubble" on the pre-assessment; that is, below the passing score but within range of passing.
Equity: Each small learning community selected 15 students, including regular education students, English language learners, and students with disabilities, in order to create a representative intervention group.
Need: What Is the Focus of the Intervention?
The team analyzed the target group's answers on the pre-assessment and discovered two common areas of difficulty: recognizing evidence explicitly stated at multiple locations or with varied wording, and making inferences from selected parts of the text. Error analysis revealed that students based answers on recall rather than rereading to find evidence in the text that supported their multiple choice selection; students struggled with making inferences; students often chose the "near miss" answer—one in which the answer may be found in the text, but does not answer the question being asked; students struggled with nonfiction and informational texts; and students had low reading stamina on longer passages and passages later in the test. Based on this data analysis, the team set the intervention's goals: for each student to learn to find evidence from the text to support a multiple-choice selection, to read beyond a literal level of the text, and to become more confident with nonfiction texts and longer passages.
Strategies: How Can We Lead Students to Reach These Expectations?
- Communicated to families and students that this intervention was a positive, proactive opportunity critical to each student's success, and required them to sign an attendance and commitment agreement;
- Renamed after-school time as a 5th period in the block schedule, calling it "E Block," so that students perceived the intervention as part of the school day; and
- Provided a modest stipend (with conditions) to each student to help those for whom after-school jobs would have prevented participation, and to reinforce that the intervention was not punitive.
To provide curricular support, Brighton High's staff shared teaching strategies, such as: giving students multiple choice questions from the MCAS without the answer choices, requiring students to write out where, why, and how they found evidence supporting the answer they selected, and reading strategies like rereading chunks of text, annotating, and using the "What does it say... mean... matter?" strategy (Gallagher, 2004).
To track student progress, teachers established an interim assessment plan that included five additional formative assessments similar to, but shorter than, the pre-assessment, also designed by BPE. The state exam would be the final assessment. The plan was set, and now it was time to implement it.
Do
The three groups of 15 students met with an ELA teacher from their small learning community three times a week during the afterschool "E-Block" for eight weeks to receive explicit reading instruction. Intervention time could then be 100 percent focused on instruction. These teachers, along with the literacy coach and ELA teacher leader, collaborated frequently throughout the intervention to monitor progress and troubleshoot. Other 10th grade teachers reinforced strategies learned in the intervention and administered the same common interim assessments, in their own classes. Administrators responded immediately if student attendance faltered, checking in with students and parents and reminding them of the need for the intervention to ensure success.
Check
Throughout the intervention, students performed better with each interim assessment. Teachers noticed students involved in the intervention participating more in class, and applying the reading strategies to other classes besides ELA. A biology teacher noted, "Students who wouldn't even attempt to read the textbook are now annotating on [sticky notes]. They are applying reading strategies on their own and understanding more." Teachers felt hopeful for the results, and anticipated the positive outcomes on that year's MCAS.
Following the intervention, the ELA MCAS results for the targeted students exceeded expectations. Thirty-eight of the 45 students identified completed the program. Eighty-six percent of those students passed with a score of at least needs improvement, and of that group, thirty-six percent received the higher score of proficient. Overall, Brighton High's proficiency rate on the 10th grade ELA test had soared by 23 percentage points from the year before. In that same year, 2009, the Harvard University Achievement Gap Institute (AGI) measured the value added by all high schools in Massachusetts, comparing the 8th grade ELA scores of incoming students in 2006 with the 10th grade ELA scores of the same students two years later in 2008. AGI researchers determined that Brighton High School had outperformed 94 percent of schools in the state in creating improvement for several groups of students at the lowest performance levels. The year following the intervention, Massachusetts's newly computed Student Growth Percentile identified Brighton High as the Boston district high school with the highest growth percentile for ELA. In addition to test scores going up, students learned how to independently apply reading strategies to challenging text. Jerry Anderson told a reporter from a local newspaper, "My teacher showed us how to be confident in ourselves."
Act
It was clear that the intervention had worked. In understanding what made the initiative successful, so it could be modified and repeated the next year, the team adapted George Knight's (2007) pyramid of interventions considerations, which lists key elements for ensuring the success of school-based interventions.
Have a clear, specific goal connected to the school mission:Rather than focus on macro issues such as grades and literacy levels, the intervention had a clear and measurable short-term goal that was also high-leverage, and achieving it positively affected other student outcomes. The intervention goal to increase the number of 10th grade students who would pass the ELA MCAS aligned with Brighton High School's mission of ensuring that our students were on track to graduate by the end of their 10th grade year and were well prepared for their lives in postsecondary education and beyond.
Target students: The team selected students who scored slightly below failing on the reading pre-assessment, which predicted that they would most benefit from a short-term intervention. In the future, Brighton High would work on other interventions of different duration and nature to target other at-risk populations in the school.
Be directive: The shift in this initiative was to move from invitational support, such as drop-in tutoring, to a required experience for targeted students. Given the complexity of the lives of many at-risk students, sustained commitment from students and parents can be challenging, but administrators and teachers made it a priority by directly communicating the urgency and critical importance of this intervention.
Be clear about who is responsible: This was not an intervention done by one or two teachers. The 10th grade team, along with the literacy coach, ELA teacher leader, school partners, and administrators, each played a specific, key role. They managed logistics for processing the data on the common interim assessments, facilitated analytical discussions with teachers about student data, and helped teachers implement the next instructional steps. The team made systematic decisions and assessed the progress of the intervention.
Use research-based strategies: The literacy coach and ELA teacher leader provided ongoing support for teachers. They developed instructional literacy tools, such as ways to create skill-specific question stems for teachers, based on prominent research. Teachers gave students opportunity in their daily ELA classes to practice strategies learned in the intervention, thereby giving students practice and confidence.
Use assessments and track progress: Teachers involved in the intervention and those in the classroom assessed students throughout the eight weeks to determine areas of need, progress, and next steps. The predictive nature of the pre-assessment allowed the team to identify the correct students, and the follow-up assessments showed that the instruction provided was improving student performance.
Consider resources: Support for the intervention came primarily from repurposing existing resources. Stipends for after-school tutors were used to pay for teacher time, and the roles of administrators, teacher leaders, and the literacy coach were adapted to support the initiative. Leveraging the support of outside partners was critical, with the BPE providing assessment tools and analysis, an educational consultant, the data processing for the common interim assessments, and training in analyzing the results.
Plan for replication: The intervention described here has continued and expanded in the two years since implementation. Subsequent ELA MCAS interventions have steadily increased pass rates and improved students' reading abilities. The after-school time, "E Block," has become part of the school day for additional targeted interventions, including 9th graders who are struggling with the transition from middle school, 10th graders who have failed algebra and geometry courses, and 12th graders at risk of not graduating due to a lack of credits.
The Real Measure of Success
The success of the intervention can be attributed to Brighton High School's belief that it was their collective responsibility to identify students at risk of failing and to act quickly. Test scores went up, teachers were empowered in their ability to reach failing students, and students like Jerry Anderson, once thought unlikely to graduate high school, were back on track.