Conflicting views about performance assessment and other trends in public education were heard at a frank panel discussion held on the last day of the conference. The panelists were Allan Bloom, a parent from Montana who is concerned about the quality of public schools; Douglas Carnine, a professor of education at the University of Oregon and director of the National Center to Improve the Tools of Educators (NCITE); Eleanor Renee Rodriguez, an adjunct professor at Union Institute in Ohio and an education consultant; Richard Strong, a former teacher and an education trainer and consultant; and Cheri Pierson Yecke, a former teacher who serves on Virginia's state board of education.
Moderator Ron Brandt began by asking the panelists what they had heard during the conference that had struck them as most significant.
Yecke: [I'm concerned by] the assertion that "performance-based assessment can be perfectly reliable and valid"—coupled with the repeated denigration of multiple-choice tests. As educators, we have to get away from conveying this idea that there's something inherently bad or evil about multiple-choice tests.
[I'm also concerned by] the assertion that performance-based assessments can be objective and unbiased. That may be right: they can be. But it's going to take time to work the bugs out. Performance-based assessments are graded by humans, and humans are simply not infallible.
The public sometimes has a trust problem with performance-based assessments, because these are graded by humans and bias can slip in. And that [point] was not adequately addressed at sessions that I attended. A lot of good things were said, but what was not said was very telling.
Bloom: The conference was on performance-based assessment, was it not? Three years ago, I heard Ron Brandt run a session at a meeting of Montana ASCD. What I heard at that session was several criticisms of the bubble test, one of which was: "Teachers tend to teach to the test." And now I come to a conference the whole point of which is to teach to the test. What's going on?
My reaction to the idea of performance-based assessment, as I understand it, is that it's not a good idea for high-stakes purposes. It's a criterion test. It's not authentic. It's subjective, because it is only valid for the people who are putting it together. And those people reflect their view of validity, which may not be respected anywhere else.
There are really two problems with it. People have different ideas [about what good work is], so the thing's not fair. Secondly, I don't have any confidence in any criterion that's developed by professional educators, because educators do not and cannot justify programs. . . . If you can't answer all the criticisms and questions substantively, then the program deserves to die. Now, who in education is threatened with losing his or her job status because the thing that that person laid on the rest of the world flopped? No one. So there's no responsibility, and that shakes my confidence in what you all are doing professionally.
Carnine: There are many serious problems with standardized tests, and the education community had a great opportunity to come up with something far better, which to some degree has been squandered.
In Vermont, portfolio assessments are reliable [only] at the state level, which means you don't know about a school district or a school or a child. . . . Performance and content assessments in Arizona correlated zero, and they threw them out. California spent 30 to 40 million dollars on a state assessment that was more performance based, and then they told the legislature they couldn't give a score on any individual in the entire state. . . . [Now] California is going back to an off-the-shelf, standardized, norm-referenced test, because the educational community couldn't deliver on something that made any sense whatsoever.
We have to get our act together to produce solutions. Sitting around and criticizing is not the way out of this. The recent history of the state assessments is not very encouraging. What is encouraging is that the whole movement toward state assessment is criterion-referenced, where you try to talk about kids' meeting a standard for performance that's important. That's an extremely positive development, and I just hope we don't screw it up. But we're doing a pretty good job of screwing it up by having sloppy assessment practices that lead to skepticism and disbelief on the part of the public—with good reason.
Rodriguez: The title of this panel discussion is "What Is Really the Best Way to Bolster Student Learning?" And I think the way to do that is to teach. And that's been the biggest challenge. We need a C.I.A. in education. Curriculum, Instruction, and Assessment—the three pieces need to be totally connected.
We need to ask: What is the goal of the test? As Rick Stiggins said in his presentation, stepping on the scale doesn't make your weight change. So if we're talking about student learning, what are we going to do after we get those [test] results back? If we don't do something specific with the results we get, then nine times out of ten, if you keep doing the same stuff, you'll keep getting the same stuff.
I don't care if it's criterion-referenced, performance, or whatever you call it—student learning is not going to improve unless you do something as a result.
Strong: What stood out for me was a teacher from Florida who told me that last year she decided to do three Shakespearean plays with her 7th graders, 13 of whom were identified as learning disabled. This had been a remarkable year for her. What struck me was how full of wonder her voice was. It wasn't full of ideology or a will to dominate or a need to win. It was full of wonder and curiosity. . . .
If I have any sorrow about the culture of the conference, it is that this voice of curiosity—as opposed to a voice of a will to dominate—is nowhere near as prevalent as I would wish it to be. What could we do that might perhaps permit us to become more curious about others, more empathetic toward others, and less earnest about dominating and controlling the conversation?
Carnine: My agenda is defining a constructive role for science [in education] so that we can have informed decision making. [Educators] need information that's readily accessible, that's practical, that you can trust, to help you make decisions. In my opinion, we don't currently have that.
We must develop a stable knowledge base in education, the same as we have in other professions. . . . If we cannot come up with rules of evidence in education that produce routine levels of results—and that's what a profession does, it produces routine levels of satisfactory practice—then we will move more and more toward charters and vouchers. And through self-selection, parents of children who do better in school will go to these alternative programs, and the evaluations will therefore show that they work. And therefore, for the wrong reasons, vouchers and charters will turn out to be successful—if we continue in a world where we don't know anything.
And that will then be the beginning of the demise of public education, because we'll continue to quarrel and everybody will do their own thing, because we don't know if anything's better than anything else.
Strong: In the late 19th century, as science began to produce effects, people who operated in the social sphere began to imagine that something like education could be a science, and therefore there would be standards of evidence, and that evidence could then be put forward in order to guide decisions. It is not at all clear that this is true. . . . We keep confusing goals with scientifically provable statements.
Bloom: [Professional educators] won't engage in dialogue. When you get somebody like me who comes in and wants to talk substantively, who takes the trouble to try to understand what you're talking about and raise some questions about it, you slam the door in my face. [I could easily] go someplace else—and it's happening all over the country. And the reaction from our state director of public education is: "My god, you wouldn't want vouchers! Suppose somebody would want to build a school on the principles of the Ku Klux Klan!" Talk about scaring people in order to avoid justifying the substance of what you're doing! It's terrible.
Yecke: What must it take for alternative voices in education to be heard? For just one example, let's look at the teaching of reading. Everybody knows about the controversy over whole language and phonics. The point that needs to be made is this: All too often in the past when a parent has approached a teacher or principal and made it known that they want their child to have explicit phonics instruction, they were labeled as a member of the religious right, as an extremist, or maybe some kind of right-wing nut case.
And why? Because they were voicing an alternative view; they were challenging the status quo. And here we are in an era where we preach tolerance and respect for diversity, and yet alternative voices in education are often treated with derision—not even afforded the common courtesy of tolerance, let alone respect.
Strong: One of the great unsolved problems in our talk about education is: What is the role of professional conversation, and what is the role of public conversation? When is the parent the decision maker, and what knowledge base does the parent stand on? Or are there decisions that are not based on knowledge but on something else? And when is the teacher or administrator the decision maker?
Carnine: We as a profession have very serious decisions to make about how we do business. The willingness to impose a radically different approach to teaching mathematics in all the high school classrooms in the entire state of California, based on not a single shred of evidence, is the kind of capricious and dangerous decision making that could occur in no other profession. Now, this is not a problem of individuals making poor decisions. It is a problem of the education culture tolerating such slipshod behavior.
You may not be aware that teachers cannot be successfully sued for malpractice, because the courts of this country have ruled that teachers don't know anything. They have not ruled that teachers are misusing information; they are ruling that they don't know anything. And it's because the trendsetters—the people who are [providing] the knowledge base for the field—are not doing so in a way that merits trust and belief in the larger society. . . . The trendsetters are either going to join in trying to find solutions to the problems [of public education] based on some kind of research base, or they are going to continue to follow dogma and dominate with their belief system. And, believe me, that is a course for disaster.