Amazing............As Stakes Rise, School Groups Put Exams to the Test
Critics Decry Heavy Reliance On Standardized Measures
By Michael A. Fletcher
Washington Post Staff Writer
Monday, July 9, 2001; Page A01
This spring, students taking North Carolina's inaugural fifth-grade promotion test in math received an unexpected gift: School officials set the passing score too low, allowing virtually all 99,000 students to pass.
The error was a godsend for some marginal students who might have been held back. But state education officials were left to explain how a test integral to determining the futures of tens of thousands of students could be so fundamentally flawed.
Now that Congress is moving toward approving President Bush's education reform plan -- a plan that relies heavily on standardized testing -- some observers are becoming increasingly vocal in making a case against such exams. Parents in New York and other states made news this spring when they protested the reliance on tests, but now equal attention is being paid to testing experts and school officials who are questioning how tests are administered and interpreted.
Bush's reform plan, which won strong bipartisan support in both the House and the Senate, now awaits a conference committee and is likely to go to Bush later this summer. The plan would require annual testing for students in grades 3 through 8, with a lot riding on the results: Schools whose students perform poorly on tests could lose a portion of their federal aid, be reconstituted with new staff or converted into charter schools. Parents would be able to use public money to buy private tutoring for their children, or to enroll their children in better-performing public schools.
In its reliance on testing, Bush's plan reflects both public sentiment nationwide and practice in many states.
Business leaders and others have demanded that students show mastery of basic skills, and polls show that at least three of four Americans endorse using standardized testing to reward or punish students and educators.
North Carolina -- one of more than 30 states that rate schools based, in large measure, on their test scores -- is often cited as a national model of test-driven reform. It rewards teachers, penalizes students and rates schools based largely on student performance on standardized tests. In Florida, tests are the arbiters of graduation and promotion. In California, raises and bonuses for teachers and principals are determined in large measure by test results. In Alabama, school budgets can be enhanced based on test results; educators' careers can rise and fall with them.
Test scores, long a factor in real estate values, are even being cited in child-custody cases, with lawyers arguing that children should be awarded to the parent who lives in the higher-scoring school district.
Education Secretary Roderick R. Paige has called frequent testing the key element in improving education. "There is no logic in not knowing where a student is," he said. "Tests provide irrefutable data about which kids are learning what, and in which classrooms."
Testing experts, educators and parents agree that the education system needs some measure of assessing a student's performance. But some argue that standardized tests offer a defective measure -- that the tests are being misused, that testing errors are increasing as the demand for tests rises, and that the emphasis on testing is leading teachers to steal time from the curriculum to prepare their students for the tests.
Diagnostic Tool
Standardized tests "give you information about the child," said Jeffrey W. Taylor, a senior vice president at NCS Pearson, the nation's largest test-scoring company. "Also, they give you a diagnosis across the board of schools and teachers, where they are strong, where they are weak. You are then in a position to do something about it."
But critics of Bush's plan say tests are rarely used that way. Of the nation's Title I schools -- schools whose percentages of low-income students qualify them for special federal aid -- fewer than half identified by test results as needing improvement actually received additional help, according to a federal report released earlier this year.
"The instrument itself may be capable of this kind of diagnosis, but most school systems have no experience in using data in a real-time way to inform instruction," said Steven Hodas, executive vice president of the Princeton Review, a firm that prepares students as young as fourth-graders for standardized tests. "The tests are not used as a snapshot; they are a goal."
Beyond that, the critics say, standardized tests provide an incomplete -- and sometimes distorted -- measure of student performance. Students can score well on one standardized test and poorly on another. They sometimes improve their test performance as they become more familiar with the measures, or do poorly because of distractions inside and outside the classroom.
Even good performance on tests may not indicate that students are learning. Preliminary results from a study to be published next year by the Rand Corp., a California research group, found that 50 percent to 80 percent of the improvement in a school's average test scores from year to year is related to random factors rather than to real gains in learning.
Because of those intrinsic problems, say test publishers and research groups, including the American Educational Research Association, decisions affecting life chances or educational opportunities of individual students should not be made on the basis of test scores alone.
Supply, Demand and Error
As standardized tests have become more common, and as the consequences they trigger have risen, critics say, there have been more testing errors -- not only in scoring the tens of millions of tests students take each year, but also in setting passing standards.
The highly specialized business of writing and scoring standardized tests is dominated by three major test developers and one major scoring and data management firm. As demand for testing has grown, they have been under enormous pressure to expand rapidly to keep pace.
Overall, the testing companies have tiny error rates. But their errors, when they occur, can affect thousands of children's lives and educators' careers.
In 1999, an error by National Computer Systems Inc. (now NCS Pearson) resulted in 8,000 Minnesota students being told they had failed a high school graduation test they had actually passed. Before the mistake was discovered, 48 high school seniors had been wrongly denied their diplomas.
Two years ago in New York City, a scoring error by CTB/McGraw-Hill caused nearly 9,000 students to mistakenly be sent to summer school.
Testing company officials say some scoring errors are caused not only by the huge increase in volume but also by deadline pressures -- states and school districts want to test as late in the year as possible but receive results before students go on summer vacation.
"When you speed up processes, you increase the risk of there being errors," said Jon Twing, manager of psychometric services for NCS Pearson, which expects to score 40 million standardized tests this year. "You just don't have the time to do the quality control."
Aside from outright scoring errors, testing opponents say, the standards for student proficiency differ from state to state, and state officials can err in setting those standards, as North Carolina's recent problem underscores.
"The percentage of students scoring at the 'proficient' level on state assessments varies widely across states -- differences which do not necessarily reflect actual differences in student achievement," said a report released in January by the U.S. Department of Education.
For example, 26 percent of fourth-graders were rated at least "proficient" on Tennessee's state-administered reading test in 1998. An almost identical percentage was rated proficient on the reading portion of the federal government's widely respected National Assessment of Educational Progress (NAEP).
But in the same year, 89 percent of Texas fourth-graders were rated proficient on that state's reading test, while just 29 percent were rated as proficient on NAEP.
Similarly, 9,100 schools that receive special funding for disadvantaged students are said to be in need of improvement as determined by their standardized test scores. But the number of such schools varies from state to state, depending on each jurisdiction's educational standards. In Texas, just 1 percent are deemed in need of improvement; in Michigan, 76 percent carry that designation.
"It just doesn't make sense," said Bob Chase, president of the National Education Association, the nation's largest teachers union. "If you are going to use these results as benchmarks to decide what kind of money will go to school districts and states, there has to be some degree of comparability, and right now there isn't."
Where the Time Goes
The emphasis on testing has led to cheating scandals, including two recent episodes in Potomac and Silver Spring. But critics of testing contend that even when there is no outright cheating, the importance attached to standardized test scores is pressuring teachers to focus too much classroom time on them.
Some teachers in Houston complain that music and art instruction have been sacrificed, and in some places, critics say students spend as much as a month preparing for and taking standardized tests.
In Pittsburgh, teachers say they have to forgo some field trips and advanced writing instruction. "The entire focus of school changes for one, two, three months because of the tests," said Melissa Butler, a second-grade teacher at Knoxville Elementary School there.
Even in North Carolina, there is some dissatisfaction over the testing program.
"At our school, we have been trying to develop ways for kids to get more time in art, drama and music and maybe integrate some of those areas into the core academic subjects," said Malcolm Goff, an art teacher at E.K. Powe Elementary School in Durham, N.C. "But we knew that if we had the kids doing that and the test scores went down, the change would be blamed for the decline. So we didn't do it."
In a small but growing number of school districts, from New York to California, parents have led boycotts of state-mandated exams, contending that high-stakes tests lead to simple-minded reform.
"We're not opposed to testing. We are not opposed to high standards. But we are against high-stakes testing," said Jane Hirschmann, a Manhattan parent who has led boycotts in New York. "This is not about kids, it is about politics. A test is a measure, it is not a reform."
That revolt, coupled with the prospect of high failure rates, is resonating in some states.
Last month, Ohio eliminated an impending requirement that students who don't pass a fourth-grade reading test be prohibited from advancing to fifth grade. The state also replaced its high school exit test with a series of end-of-course tests.
In the past year, Arizona and Alaska have postponed the effective dates of their high school graduation tests.
Even North Carolina is making changes. After parents, teachers and legislators complained that students spent as many as 30 days -- one-sixth of the school year -- preparing for and taking standardized tests, the state recently eliminated three of the exams. |
