by Gabriel A. Reich, Assistant Professor, Virginia Commonwealth University, Department of Teaching and Learning.
Since the Reagan administration, Americans have listened to a steady drumbeat decrying the failure of our public schools. We demand action and politicians respond requiring administrators, teachers, and students to be held accountable for results. No Child Left Behind (NCLB) was designed to make accountability in education national policy. It requires states to test all children in math and literacy, and to attach high-stakes sanctions for those who fail to make the grade. The most unpopular aspect of NCLB is the stipulation that schools be judged “failing” if too many students do poorly on tests. Schools around the country, even highly respected ones, are caught in a bind because the percentage of students who must pass increases every year. In 2014, the goal set by NCLB is that 100% of students will be “proficient” in math and literacy or their schools will be labeled failing. As the law approaches this statistically impossible end, the Obama Administration and the Commonwealth of Virginia have decided to shift focus of accountability away from schools to individual teachers. This shift will exacerbate rather than ameliorate the problems caused by NCLB.
To hold teachers accountable for results, and thus improve teaching and learning in the Commonwealth, the Virginia Department of Education has required all districts to devise a method for evaluating the effectiveness of teachers. These evaluations will be used to make decisions about which teachers will be awarded an annual cash bonus. Evaluations must base 40% of a teacher’s effectiveness score on the results of “standard measures,” i.e. SOL exams.
The failure of this policy is predictable because it is based on two ideas that are completely divorced from reality. The first is the idea that standardized-test scores are so accurate we can base judgments about the effectiveness of individual teachers and the learning of individual students on them. The second delusional idea is that top-down accountability policies will improve the state of teaching and learning.
Americans’ faith in what standardized tests can accurately measure is completely out of proportion to what they were designed to measure. Standardized tests rely overwhelmingly on multiple-choice questions, a technology that is cheap and efficient but produces inaccurate results. Second, even a well constructed test produces more useful information at the aggregate level, such as a school, a district, or a state, than at the level of individual students and their teachers. Thus, a third grade SOL reading test might provide useful information for comparing Henrico County elementary schools to those in Hanover County, but they are fairly useless for comparing how “good” at literacy your child is compared to another. This is why the American Psychological Association and the American Educational Research Association both have position papers against imposing high-stakes on the test results of students and their teachers.
Faith in testing underwrites an equally damaging belief in “accountabilism.” Accountabilism is the belief that institutions, such as schools, will improve only if you demand that everyone in them prove that they are doing what they are supposed to do. Accountabilism assumes that all the necessary tasks in an institution can be identified, and that there is a single best way to accomplish each of these tasks. The kind of surveillance needed to ensure that all these things are being done the right way is massive, if not impossible. Thus, policy makers look for proxy measures for these complicated processes and use them to make decisions about whether people have done a good job or not. For example, it is both expensive and time consuming to train enough inspectors to visit every teacher a couple of times a year to see how good their pedagogy is. It is much cheaper to use standardized tests, which taxpayers have already paid for, as a proxy for everything that happens in schools. This is akin to licensing drivers using only a multiple choice test. The test might correlate very highly with people’s driving ability, meaning good drivers tend to get higher scores than bad drivers, but I for one would not feel safer on the road knowing this.
Accountabilism may make logical sense in the abstract, but it falls apart when faced with the reality of “Campbell’s Law.” Campbell’s Law states that as the stakes tied to a quantitative measure are increased so is the likelihood that stakeholders will game the system in order to appear effective. These actions produce inaccurate results and corrupt the phenomenon being measured. This is why cheating scandals have increased along with the popularity of test-based accountability. Cheating is a fool-proof method of achieving the goal that has been set out for you – improving test scores in a short period of time.
As Virginia’s teacher-merit-pay policy is enacted, Campbell’s Law will inevitably kick in further distorting the work of teachers. As has been shown in the districts that have adopted value-added systems, the use of test-score growth to measure teacher effectiveness is unreliable, leading to seemingly random results, lower teacher morale and an exodus of people from the profession. Statistically, test score growth is most likely for those who teach middling, middle-class students. Those teaching poor, low-achieving, disabled and gifted children will be denied cash bonuses no matter how well or how hard they work because these populations are less likely to show a raise in scores on standardized tests from year to year. Even more damaging, is the way this policy will pervert the relationships between teachers and their students. Students will hold a new power over their teachers through their test scores, and teachers will be in a position where they are likely to resent their students for not making the gains needed to earn a bonus. The policy will make school-wide improvement less likely because it provides a disincentive for teachers to work together in teams to improve their practice.
What is not inevitable is that teachers, parents and taxpayers will blindly accept so wrongheaded a policy.