Main image
6th June

First Published in The New York Sun, June 6, 2003

By Andrew Wolf

Many of the plans advanced by Mayor Bloomberg and Schools Chancellor Klein depend on the data provided by highstakes tests. But can Messrs. Bloomberg and Klein — not to mention the public — depend on these scores and the way that they are currently disseminated and analyzed?

Our school system has stumbled into a Balkanized system of testing. The State of New York administers tests in the fourth and eighth grades in English and math. The city gives its own, entirely different tests in the third, fifth, sixth, and seventh grades. This makes tracking the progress of individual children as they proceed through the system quite difficult. That the city is using a new testing device this year — one from Harcourt Educational Measurement, rather than the CTB/McGraw Hill test previously used — makes things even more complicated.
While the city’s tests are all multiple choice,the state tests include essays and math problems that need to be scored by hand using a “rubric.” It is my understanding that the subjective portions of the tests are graded by district office personnel, a questionable practice.

These raw scores are then translated by some unknown process into a “scale score,” which is, in turn, converted into the familiar levels 1, 2, 3, and 4, which go from “do not meet the learning standards” to “exceed the learning standards.” Despite the fact that the testing devices are so dissimilar, the results are sorted out into one continuous scale.

This part of the process is entirely subjective. How many correct answers translates into Level 4? How does this break down for each of the grades? Setting the bar one correct answer higher or lower could move thousands of students into one category or another.Who makes this decision? Can this be manipulated in subtle, but significant, ways?

I don’t have the answer to this question, nor am I accusing anyone of cheating or manipulation. But these decisions, like everything about our education system, must be made with total transparency.This is the way the people who administer the SATs conduct themselves. Public school students and their parents deserve nothing less.The scoring and “norming” of the tests ought to be done by a professional panel totally independent of local school boards and political influence.

But there is an even larger problem with our testing system. Last February, the schools chancellor announced a list of 208 schools that will be exempt from the city’s uniform curriculum — a list that since has been expanded. These schools were chosen largely on the basis of test scores.Not surprisingly,therefore, a disproportionate number of these schools are located in middle- and upperincome areas, such as District 2 on Manhattan’s Upper East Side and District 26 in Queens, which includes upscale areas like Bayside and Douglaston.

Do the high scores of these schools reflect better instruction, or are the scores a reflection of favorable demographics? Maybe some of these highscoring schools could be doing even better using another instructional model. Because of the way we are using the data collected by the tests, we will never know.
The tests will also seal the fate of the 50 “low-performing” principals that the schools chancellor has vowed to dismiss. But one might well find some of the worst principals in the city in some of the highest-scoring schools, and some of the best in the schools that seem to perform the worst. There is no trick to getting high scores when your school is filled with the children of professionals making six-figure incomes — children who typically come to kindergarten already knowing how to read. The real measure of talent is how far you can bring each individual child
from whatever point he or she began the school year.

That is why it might be worth adopting a new approach. Measuring the average growth of children is known as “value-added” testing. It recognizes that each child may start out in a different place, but it is how far they progress that is the true measure of success. Rating a school, a principal, a teacher, or even a student by the percentage of students at or exceeding grade level misses the point. Every year, the average student should gain a full year on a uniform scale. If children who start out at a high level gain only a half a year, they may still outscore children who started out at a lower level and picked up two years. Which school, which principal, which teacher is doing the better job?

This methodology has been adopted in many school districts across the nation. It is hard to believe that it won’t be embraced by many more. It is this kind of reform that makes every other initiative possible.
Before we can evaluate the job our teachers do, before we can rate our principals or analyze the efficacy of this reading program or that, we need the best possible data. We aren’t getting it now, and this calls into question every decision made by the mayor and chancellor in their quest for change.

© 2003 The New York Sun, One, SL, LLC. All rights reserved.


Leave a Reply