Published: March 18, 2006 | By KAREN W. ARENSON | New York Times

As the College Board races to score the final 1,600 exams from its problem-ridden October SAT test, a string of recent testing errors around the country has college and high school officials, testing experts, students and parents asking with rising urgency, Who is watching the testing industry?

Spurred largely by the No Child Left Behind law, testing has exploded in recent years. Educators are now trying to measure factors like whether toddlers in Head Start know their letters and whether elementary and high school students are making progress in reading and math.

The states alone are administering about 45 million tests this school year.

And for students headed to college, there are the Advanced Placement exams, the SAT and the ACT.

The volume is stretching the $2 billion-a-year education testing industry, educators say, taxing its ability to draw up enough tests and score them quickly and reliably. The states are struggling to find experienced officials to provide quality control.

The resulting flurry of errors has educators and lawmakers calling for better disclosure and oversight. Some are even proposing a national agency like the Food and Drug Administration to regulate testing.

"We need accountability," said George F. Madaus, a research professor at the Center for the Study of Testing, Evaluation and Educational Policy at Boston College. "I certainly wouldn't get rid of testing, but we need to be much more aware than we are now about the shortcomings, the limitations and the fallibility of the technology."

The past few weeks have shown the range of the problems. The College Board's disclosure that at least 4,600 students out of 495,000 who took its October SAT test had scoring errors was followed this week by an $11 million settlement by the Educational Testing Service of a case involving scoring errors for 27,000 people on tests that more than half the states use for teacher certification.

The Illinois superintendent of education, Randy J. Dunn, threatened last week to recommend ending the state's $44.5 million five-year contract with Harcourt Assessment because state tests were delivered late with misprints and collating errors. (He later backed away.) And New York State said this week that its seventh- and eighth-grade math tests had several questions that had been in test preparation materials.

"All of these tests have errors," said John Katzman, chairman and founder of the Princeton Review, a test-coaching company that has benefited from the testing boom. "The questions might be flawed in some way. The scoring might be flawed. The administration is often flawed."

A large part of the problem, policy makers and educators say, is that the demand for tests is outstripping the abilities of the industry, and testing and scoring quality have deteriorated. Adding to the pressure, each state has its own specifications.

Robert L. Linn, a professor emeritus at the University of Colorado at Boulder who serves on a College Board advisory committee, said that problems like those with the October SAT were not uncommon in testing.

"These things happen now and then in a lot of testing programs," Dr. Linn said. "But there are too many now; the industry is stretched pretty thin." He added, "It's pretty clear, I guess, that the quality-control issues need to be looked at again."

In a recent study for EducationSector, a new education research organization, Thomas Toch, the group's co-director, found that the high stakes of the No Child Left Behind law, which sanctions schools that do not improve, had states trying to administer tests as late as possible so children had the most time to prepare. But they still want scoring completed in time for summer school placement, giving companies little turnaround time.

Mr. Toch said states were having trouble recruiting and retaining experts, leaving testing companies largely responsible for their own quality control. Companies, too, are struggling with these demands.

"They get so little money for these contracts that they are hard pressed to hire all the people they need to do this immense amount of work without making mistakes," he said.

Joyce E. Karon, a member of the Illinois State Board of Education, said that although she believed Harcourt was "a reliable company," like other test providers, it was "working at the brink, and when you work at the brink, things happen."

Mrs. Karon said that before the board chose Harcourt, it asked other states whether they had experienced problems. "Almost universally," she said, "they had all had some problems."

Rick Blake, a spokesman for Harcourt Assessment, said the company regretted the hardship that the problems had created for Illinois and was working to ensure that the rest of the testing went "smoothly and without further delay."

Still, as the pressure for more tests intensifies, testing experts and policy makers are beginning to weigh whether more oversight is needed.

One suggestion would require that testing problems be reported publicly. W. James Popham, an emeritus professor at the University of California at Los Angeles, said that more exposure might help. "Frankly, because it is in the best public-relations interest of both the scoring service and the state officials who hire them, many scoring muck-ups are masked," Dr. Popham said.

One step toward disclosure could come from the inspector general of the federal Education Department, who is planning to study what states are doing about errors and whether there is need for federal oversight.

Another approach being suggested by Mr. Toch, Dr. Madaus and others is to have some kind of auditor or oversight board, an independent entity or a federal body. "There are all kinds of things in society that get monitored," Dr. Madaus said. "Nobody is seriously looking over the shoulder of those testing programs."

But some in the industry say regulation is unnecessary and will raise the price of testing. "When something like this occurs, you want to make sure you don't create regulations or ways of doing things that increase costs and don't improve the quality of service," said Gaston Caperton, the College Board president.

Kenneth P. LaValle, the New York State senator responsible for the state's 1979 Truth in Testing law mandating disclosure, said he believed more disclosure of test questions and answers might be needed.

"We are now living in a testing culture," said Mr. LaValle, Republican of Long Island. "We need accuracy and security and all these things."

Some counselors are urging college-bound students to pay extra to obtain answers after a test is administered to check their own performance, and for hand scoring.

Eugene Falik, a computer specialist in Far Rockaway, N.Y., said that after his daughter signed up recently for an Advanced Placement exam, he spotted the hand-scoring option. "The clear implication is that these marked sense sheets, or optical mark readers, are not reliable," Mr. Falik said.

More colleges have stopped requiring the SAT. Joanne V. Creighton, president of Mount Holyoke College in Massachusetts, said she felt her college's decision five years ago to make the tests optional was reinforced when she learned of the SAT scoring errors. She said the college had found this made "no measurable difference" in quality.

But others say testing is a crucial tool. Richard P. Mills, education commissioner in New York State, said recent problems had not changed his mind about the usefulness of testing. "It can be done right," Mr. Mills said. "Does that mean it's flawless? No. Errors crop up. No error is acceptable. But testing is indispensable."