Chapter 1. The Changing Face of Educational Assessment

Educators, legislators, business leaders, and parents recognize that educational goals should reach far beyond the scope of traditional subject-area domains. For example, in the report What Work Requires of Schools (Secretary's Commission 1991), the Department of Labor identifies a broad array of both academic and nonacademic competencies as necessary for the modern workplace, including:

• creative thinking
• decision making
• problem solving
• learning how to learn
• collaboration
• self-management

This emphasis on goals outside the traditional content disciplines is also reflected in some popular approaches to school reform not necessarily associated with the America 2000 proposal or its related efforts. For example, William Spady, an advocate of outcome-based education, argues persuasively that educators must broaden their target to include outcomes that relate to lifelong learning (Spady 1988).

Given these recommendations, many districts, and even entire states, have focused their attention on identifying important "exit outcomes" or "learner outcomes." For example, the Aurora Public Schools in Aurora, Colorado, have identified five lifelong learning outcomes and their related proficiencies (Redding 1991):

Outcome 1: A Self-Directed Learner

• Sets priorities and achievable goals
• Monitors and evaluates progress
• Creates options for self
• Assumes responsibility for actions
• Creates a positive vision for self

Outcome 2: A Collaborative Worker

• Monitors own behavior as a group member
• Assesses and manages group functioning
• Demonstrates interactive communication
• Demonstrates consideration for individual differences

Outcome 3: A Complex Thinker

• Uses a wide variety of strategies for managing complex issues
• Selects strategies appropriate to the resolution of complex issues and applies the strategies with accuracy and thoroughness
• Accesses and uses topic-relevant knowledge

Outcome 4: A Quality Producer

• Creates products that achieve their purpose
• Creates products appropriate to their intended audience
• Creates products that reflect craftsmanship
• Uses appropriate resources/technology

Outcome 5: Community Contributor

• Demonstrates knowledge about his or her diverse communities
• Takes action
• Reflects on role as a community contributor

Similarly, the state of Kentucky has identified six broad learning goals as the foundation for its ambitious educational reform agenda (Kentucky Department of Education 1991). These learning goals for students combine traditional academic outcomes with lifelong competencies necessary for success beyond the schoolhouse walls:

Goal 1: Apply basic communication and mathematics skills in situations similar to what they will experience in life

Goal 2: Apply core concepts and principles from mathematics, science, social studies, arts, and practical living studies, and vocational studies to situations similar to what they will experience in life

Goal 3: Demonstrate self-sufficiency

Goal 4: Demonstrate responsible citizenship

Goal 5: Think and solve problems

Goal 6: Integrate knowledge across disciplines

In short, educators are increasingly seeing the need to identify standards for both traditional discipline knowledge and outcomes that relate to lifelong learning. Although educators have acknowledged the importance of reinforcing standards and outcomes like those listed above, they have quickly recognized that current assessments do not adequately address these standards and outcomes. For the most part, standardized tests require students to recall or recognize fragmented and isolated bits of information. They rarely ask students to apply that information, and they almost never require students to exhibit proficiencies in the "higher forms" of cognition, such as complex reasoning and self-directedness (Marzano and Costa 1988). Lauren Resnick characterizes the problem in this way:

Many of the tests we do use are unable to measure what should be the hallmark of a "thinking" curriculum: the cultivation of students' ability to apply skills and knowledge to real-world problems. Testing practices may in fact interfere with the kind of higher order skills that are desired (Resnick 1987, p. 47).

These shortcomings make it clear that new approaches to assessment are needed if we are to satisfactorily assess students' ability to meet the lifelong learning standards and demanding content standards and outcomes that are the centerpiece of the plan to make the American public education system the best in the world.

The Relationship Between Assessment and Teaching and Learning

A second factor contributing to the need for assessment reform involves the relationship between assessment and the processes of teaching and learning. Behavioral theories that characterize learning as the accumulation of discrete skills have given way to a conception of learning and teaching based on cognitive psychology. Lori Shepard summarizes this shifting conception of the learning process:

The notion that learning comes about by the accretion of little bits is outmoded learning theory. Current models of learning based on cognitive psychology contend that learners gain understanding when they construct their own knowledge and develop their own cognitive maps of the interconnections among facts and concepts . . . . Real learning cannot be spoon-fed one skill at a time (Shepard 1989, pp. 5-6).

As Shepard's comments imply, this cognitive view calls for an active, constructive approach to learning in which the whole is greater than the sum of the parts. This holistic view of learning is reflected in contemporary instructional methods such as integrated language arts; "hands-on, minds-on" approaches in science; writing-to-learn across the curriculum; problem solving and reasoning emphases in mathematics; and cooperative learning. If learning occurs in a holistic fashion, then assessments, too, should be able to provide holistic information, not just bits of information. This shift in emphasis is made explicit in a report from the National Commission on Testing and Public Policy:

If we want students to learn how to solve open-ended science problems, we should assess their problem-solving skills by other means than multiple-choice tests in which they choose among alternative prescribed answers. Carefully crafted assessments would ask students to supply answers, perform observable acts, demonstrate skills, create products, and supply portfolios of work (National Commission on Testing and Public Policy 1991).

Conventional selected-response test formats (e.g., multiple choice, true/false, matching) are quite narrow in their focus. They provide only a snapshot or a "one moment in time" picture of learning. Although such sampling may have certain uses, it is generally incapable of revealing in any comprehensive way what students know and can do. Moreover, the conditions of such tests are often highly controlled. Students complete the work within inflexible time limits and have restricted access to resources and limited opportunities to make revisions. These kinds of tests also sacrifice authenticity, since they differ markedly from the ways in which people apply knowledge in the world outside of school. Despite these limitations, the results of such one-time measures are frequently used to make significant decisions, such as whether a student should be admitted to or excluded from special programs and what final grade a student will receive in a class.

What we now know about learning indicates that assessment and learning are closely and intimately tied. The importance of changing assessment practices so they mirror the learning process becomes more clear when one realizes that students in American schools learn what they know they will be tested on. This behavior was dramatically brought to the public's attention in 1983 in Walter Doyle's commissioned paper for the now-famous report A Nation At Risk (National Commission on Excellence in Education 1983). Doyle found that students in American schools soon discover that all things learned are not equal: you are tested on some and not on others. Not surprisingly, most students choose to ignore those things on which they are not tested. And teachers tend to consciously or unconsciously focus their instruction on the learnings that are prescribed and tested by the school, district, or state (Doyle 1983). In effect, then, assessment has both a direct and an indirect effect on learning. Assessment directly affects learning in that it provides the necessary feedback for effective learning. It indirectly affects learning in that instruction is commonly skewed toward what is assessed; and, obviously, what is taught affects what is learned.

Given our new understanding of the learning process and the relationship between assessment and both teaching and learning, there is little doubt that reformed assessment practices are long overdue.

The Limitations of the Present Ways of Recording Student Performance and Reporting Credit

A third factor driving assessment reform is the manner in which achievement data are recorded and reported. Critics point out that the current methods do not provide meaningful feedback about student performance. For example, at the secondary level, the majority of American school districts rely on the Carnegie unit, which is based on a specified number of clock hours. In other words, students receive credit for the amount of time they spend in specific classes. This system has one major advantage: it can easily be standardized across schools, districts, and states. However, it also has the negative effect of rewarding "seat time" rather than demonstrated competence. All too many educators can cite specific examples of students who received credit for a course primarily because they showed up every day, not because they acquired any new knowledge or skills. A number of advocates for assessment reform are calling for a modification of this certification procedure to emphasize the role of demonstrated proficiency judged against established performance standards (Wiggins 1991).

The problems inherent to the Carnegie unit are also found in classroom record keeping and reporting practices. At the classroom level, grades are the most common means of reporting a student's performance. Course grades are generally calculated by averaging the results of various measures. Unfortunately, this approach can too often result in a distorted picture of a student's true proficiency, because specific strengths and weaknesses are masked by the process of averaging. One student with a grade of "C" might perform very differently from another student with a grade of "C." And the letter grade by itself fails to suggest ways for students to improve their performance. What does a "C" grade tell a teacher or, more important, a student about how that student can improve?

An alternative approach to classroom reporting documents students' grasp of specific knowledge and skills. Because such a system is grounded by specific performance criteria, reports are more informative and meaningful to students, teachers, parents, and the general school community. If we wish to improve learning, not simply measure it, then we must re- consider our record-keeping and reporting mechanisms.

In summary, then, a revolution in assessment is necessary given (1) the changing nature of educational goals to encompass a broad array of academic and nonacademic competencies, (2) the need for assessment practices to enhance the learning and teaching processes, and (3) the need for record-keeping and reporting systems to provide accurate and useful information concerning students' mastery of specific knowledge and skills. Indeed, such a revolution is currently under way in the form of an emphasis on performance assessment.

What Is Performance Assessment?

This book is about performance assessment from the perspective of a specific model of teaching and learning, Dimensions of Learning. Although there is growing agreement on the need to reform assessment practices, no such consensus exists regarding assessment terminology. The terms alternative assessment, authentic assessment, and performance assessment are all used in discussions of assessment reform. Although these terms are sometimes used synonymously, they have different meanings. The term alternative assessment applies to any and all assessments that differ from the multiple-choice, timed, one-shot approaches that characterize most standardized and many classroom assessments. The term authentic assessment, popularized by Grant Wiggins (1989), conveys the idea that assessments should engage students in applying knowledge and skills in the same way they are used in the "real world" outside of school. Authentic assessment also reflects good instructional practice, so that teaching to the test is desirable. Performance assessment is a broad term, encompassing many of the characteristics of both authentic assessment and alternative assessment (Mitchell 1992).

In this book, performance assessment refers to variety of tasks and situations in which students are given opportunities to demonstrate their understanding and to thoughtfully apply knowledge, skills, and habits of mind in a variety of contexts. These assessments often occur over time and result in a tangible product or observable performance. They encourage self-evaluation and revision, require judgment to score, reveal degrees of proficiency based on established criteria, and make public the scoring criteria. They sometimes involve students working with others. In Chapter 3, we describe our model of performance assessment in detail. It is important to note from the outset, however, that an emphasis on performance assessment does not imply that we should abandon conventional testing. Instead, it reflects the belief that certain educational outcomes cannot be adequately assessed through conventional formats. Indeed, the current emphasis in performance assessment supports the practices that good teachers have always used to assess and improve learning: an array of data-gathering methods, including objective tests, observations, products, performances, and collections of student work.

The Role of Standards

Readers familiar with the current national discussion of standards-based education have no doubt noticed that our discussion of assessment reform has been laced with references to standards. Standards-based education is a rapidly growing movement within the larger movement of educational reform. It is intimately tied to performance assessment. In brief, standards-based education calls for a clearer identification of what students should know and be able to do. The emphasis on clearer educational goals stems from the research finding that what students are taught in a specific subject and at a specific grade level varies greatly among schools, and even among classrooms within a school. Indeed, this was the basic finding of many of the school effectiveness studies of the 1970s. For example, Fisher and his colleagues (1978) reported that one elementary school teacher who was observed for more than ninety days taught nothing about fractions, despite the state mandate to teach the topic at that grade level. When asked about the omission of this topic, the teacher responded, "I don't like fractions." Similarly, Berliner (1979) reported a range of over 4,000 minutes (a low of 5,749 minutes and a high of 9,965 minutes) in the time spent on reading instruction in four 4th grade classes. Again, teacher preference was the basic reason behind the variation.

The standards-based education movement grows out of the assumption that the only way to ensure that all students acquire specific knowledge and skills is to identify and teach to expected levels of performance for specific knowledge and skills. Schools and districts that have embraced the standards-based movement have made a concerted effort to identify critical knowledge and skills. These efforts, however, have disclosed a number of basic issues that must be confronted by anyone engaged in standards-based education. We will briefly consider three of these issues. (For a detailed discussion of the issues regarding standards-based education, see Marzano and Kendall 1993.)

Curriculum Standards Versus Content Standards

Clarifying the distinction between curriculum standards and content standards is a basic issue in standards-based education. Curriculum standards, sometimes referred to as program standards, are best described as the goals of classroom instruction. Content standards, also known as discipline standards, comprise the knowledge and skills specific to a given discipline. Here are two standards from Curriculum and Evaluation Standards by the National Council of Teachers of Mathematics (1989):

1. develop spatial sense
2. describe, model, draw and classify shapes

The first standard describes a skill or ability a person might use in completing everyday tasks or academic problems. While driving a car, for instance, a person might use spatial sense to judge the distance of an upcoming turn. In mathematics class, a student might use spatial sense to solve a perplexing problem. The second standard is of a different sort because one does not usually model, draw, and classify shapes to complete everyday tasks or solve academic problems. Such activities are used more as instructional devices to help students understand shapes or demonstrate their understanding of shapes.

Standards like the first one above are called content standards because they describe information or skills essential to the practice or application of a particular discipline or content domain. Standards like the second one above are called curriculum standards because they identify the curricular or instructional activities that might be used to help students develop skill and ability within a given content domain. To a great extent, curriculum standards describe the instructional means to achieve content standards.

Knowledge and Skills Versus Performance

Some theorists describe standards in terms of knowledge and skills; others describe standards in terms of performance on specific tasks. For example, Albert Shanker, president of the American Federation of Teachers, defines a standard as "what we want youngsters to know and be able to do as a result of their education" (Shanker 1992, p. S11). For Shanker, identifying a standard involves identifying specific information or skills that must be mastered to gain expertise in a given domain. Diane Ravitch (1992), a former Assistant Secretary of Education, also describes standards from an information and skill perspective. Grant Wiggins, on the other hand, defines standards more in performance terms. For Wiggins (1989), a standard is a real-world, highly robust task that will, ideally, elicit or require the use of important knowledge and skills in various content domains. The emphasis on performance as the critical feature of a standard is also shared by the psychologists Shavelson, Baxter, and Pine (1992, p. 22), who state that standards should be "based on students' performance of concrete, meaningful tasks."

Content Standards Versus Lifelong Learning Standards

Content standards are those that refer to knowledge and skills belonging to a particular discipline. For example, a standard such as "understands and applies basic principles of number sense" is a content standard because it applies only to mathematics. A standard such as "makes and carries out effective plans," however, is not specific to any content area. In fact, it is not even specific to academics; it is a skill that can be used in virtually all aspects of life. This kind of standard is a lifelong learning standard because it is specific to no one discipline and can be used in many situations throughout a person's lifetime.

Standards, Performance Assessment, and Dimensions of Learning

In this book, we have taken a fairly specific position on the issues discussed above. First, we use the term standard to refer to knowledge and skills as opposed to instructional activities that should occur in the classroom. Thus, curriculum standards are not included in our standards. For us, the instructional implications of the Dimensions of Learning model represent de facto curriculum standards.

Second, we do not include performance standards in our definition of a standard. In effect, what we call performance tasks are a form of performance standard. In our model, teachers assess students by asking them to complete performance tasks that require them to meet identified standards. Standards and performance tasks maintain a symbiotic relationship. They are two interdependent and necessary components of a comprehensive assessment system. Marc Tucker, Co-director of the New Standards Project, describes the necessary integration of standards and performance tasks:

You can't assess kids' performance unless you give them the tasks, and you can't assess their degree of achievement unless they actually perform the tasks.

But first you must be clear about what you want kids to know and be able to do. Those standards become the target for creating the assessment (Tucker 1992, p. 3).

Finally, we separate standards into two broad categories, content standards and lifelong learning standards. Content standards deal with the academic knowledge and skills belonging to a specific discipline. Lifelong learning standards deal with knowledge and skills that cut across all disciplines and are applicable to life outside the classroom.

Identifying specific content and lifelong learning standards is an important part of the assessment revolution in American education; it is a task we will tackle in the remainder of this book. Remember, however, that this task goes hand in hand with planning and delivering instruction that takes into account the constructivist nature of learning, a task which the Dimensions of Learning Teacher's Manual was designed to help teachers accomplish.