Derek Neal and Diane Whitmore Schanzenbach (University of Chicago) discuss their recent paper, “Left Behind by Design: Proficiency Counts and Test-based Accountability”.
Roughly two decades ago, education policy makers in the United States began to rely more heavily on standardized test scores as performance metrics for teachers and schools. During the late 1980s and through the 1990s, many states adopted test-based accountability systems that spelled out rewards and sanctions for teachers and principals as a function of the performance of their students on standardized tests, and when the federal government adopted the No Child Left Behind Act (NCLB) of 2001, test-based accountability became a nation-wide policy. The proponents of this development cite the need to bring “business practices” into public schools and the need to make schools “data driven” in ways that mirror the practices of private-sector companies.
…
In recent work [1], we explore a different effect of test-based accountability systems on the allocation of teacher effort, and we find evidence consistent with the hypothesis that test-based accountability systems not only shape decisions of teachers concerning what to teach but also whom to teach. We show that even though advocates of NCLB offered it as a remedy for disadvantaged children who receive poor service from their public schools, the design of NCLB almost guarantees that the most academically disadvantaged children will not benefit from its implementation and may actually be harmed.
…
Recently, the federal Department of Education allowed a few states to calculate AYP based, in part, on growth in student achievement during a school year rather than levels only. This is a step in the right direction, but without careful design work, these systems may simply create a different set of unintended effort distortions among teachers. Knowledge does not come on a natural scale, and given any particular scale, gains of a given size may be easier to achieve at some points on the scale than others. Thus, something as apparently pedestrian as the scaling of exams could have significant and unintended consequences for the allocation of teacher effort to different types of students if states do not carefully design value-added versions of AYP.Because teachers are charged with fostering knowledge, character, and other things that are hard to measure, it is not obvious that incentive systems built around objective performance measures are even desirable strategies for monitoring teachers. Test-based accountability systems have nonetheless enjoyed strong support because school principals and others who monitor the performance of teachers in public schools are seen as agents of large bureaucracies that, especially in cities, have a long record of disappointing results. Nonetheless, our empirical results and the insights gained from research on the economics of organizations suggest that policy makers must tackle difficult design questions in order to construct accountability systems that deliver quality instruction for all students regardless of their aptitude and prior achievement. Policy makers should either take these design issues more seriously or follow the lead of many private sector firms and look for other ways to monitor and motivate teachers.
(Via Mark Thoma)