Designing Reliable and Valid Common Core–Aligned Math Assessments

Eric Carbaugh

The vast majority of states that have adopted math Common Core State Standards will use tests designed by either the Partnership for Assessment of Readiness for Colleges and Careers (PARCC) or the Smarter Balanced Assessment Consortium (SBAC) beginning in the 2014–15 school year. These tests will reflect key shifts at the heart of the new math standards:

• A focus on key content, skills, and practices at each grade level;
• Topic coherence within and across grades; and
• Rigor in major topics that will emphasize students' conceptual understanding, procedural skill and fluency, and application (Alberti, 2012).

At the classroom level, teacher-designed assessments must also align with the standards to ensure reliable and valid representations of student learning. Failure to do so might produce invalidity and errors among different assessments that measure the same learning goals (e.g., an exit card and a quiz each producing different results for the same goal). Such false findings regarding student performance can lead to incorrect representations of student mastery. If teachers are unsure who has mastered—or is mastering—which learning objectives, designing appropriate and responsive instruction becomes problematic. This is why validity and reliability are essential to effective classroom assessment.

Validity means the degree to which an assessment measures the intended learning targets. To explain the concept of validity, Stiggins and Chappuis (2012) suggest, "Just as we want our recorded music or high-definition TV to provide a high-quality representation of the real thing, so too do we want assessments to provide a high-fidelity representation of the desired learning" (p. 11). I often hear teachers telling students that certain assessment questions will be thrown out because "we didn't cover that." This is a validity issue. By deliberately aligning every assessment question to one or more learning goals (and also to classroom instruction), teachers can ensure their assessments measure what they intend for students to know and be able to do, without any outliers.

Reliable assessments measure the same goal with consistency; those that are not reliable will likely obtain erratic results. Assessment reliability is particularly important when considering teacher feedback for student growth. To adequately use feedback to show students that they are progressing toward higher learning levels, teachers must be able to obtain consistent results of student growth toward the same learning goals. As Stiggins and Chappuis (2012) also point out, "as proficiency improves, a reliable assessment will track those changes in proficiency with changing results" (p. 12).

For example, if a teacher were to use entrance cards to assess a student's ability to solve real-world and mathematical problems by writing and solving equations of the form x + p = q and px = q (Common Core math standard 6.EE.B.7) at the beginning of four consecutive lessons, reliable assessments (entrance cards) would yield results aligned with student improvement in each of the four days (i.e., a consistently increasing positive slope). If those assessments were not reliable, however, the teacher might notice increases and decreases in proficiency during each of the four days (positive and negative slopes).

A Strategy for Alignment

To ensure that the results of assessments are both valid and reliable, teachers should consider using a simple data-entry system to track student progress toward learning goals. The following chart, adapted from Fisher and Frey (2012), illustrates how a teacher might document individual and class error patterns to track validity and reliability on teacher-designed assessments. In addition to tracking student errors in the second and third columns, assessment items in the first column should be aligned with the expectations of the standard being measured.

Grade 3 Math Standard 3.MD.D.8: Solve real-world and mathematical problems involving perimeters of polygons, including finding the perimeter given the side lengths, finding an unknown side length, and exhibiting rectangles with the same perimeter and different areas or with the same area and different perimeters.

Note: Initials represent hypothetical students in the classroom.

The user of the chart should be vigilant that

• Multiple measures are obtaining similar results.
• Very few students display initial errors on later assessments.
• Student growth is evident and expected.

Although not an exhaustive fix to potential reliability and validity concerns, these steps help ensure that teachers measure learning goals in accurate and consistent ways. By developing valid and reliable Common Core math measures, educators stand a much better chance of appropriately responding to student needs to boost achievement on PARCC and SBAC Common Core math summative assessments.

References

Alberti, S. (2012, April). Common core state standards for math: The key shifts. Presentation given to ASCD Faculty, Alexandria, VA.

Fisher, D., & Frey, N. (2012). Making time for feedback. Educational Leadership, 70(1), 42–47.

Stiggins, R. J., & Chappuis, J. (2012). An introduction to student-involved assessment for learning (6th ed.). New York: Pearson.

Eric Carbaugh is an assistant professor of middle, secondary, and math education at James Madison University in Harrisonburg, Va.

ASCD Express, Vol. 9, No. 12. Copyright 2014 by ASCD. All rights reserved. Visit www.ascd.org/ascdexpress.