Commission on No Child Left Behind

Testing: Making It Work for Children and Schools, A Hearing at St. Joseph College, Hartford, CT, May 9, 2006

Testing: Making It Work for Children and Schools
A Hearing at Saint Joseph College, Hartford, CT
May 9, 2006

Introduction
There is broad agreement that testing plays a critical role in education reform by giving educators, administrators and the public a means to understand how schools and students are performing. However, there are important differences of opinion regarding the appropriate frequency and type of testing for measuring performance and informing instruction. There are also questions about whether state education departments and the testing industry can handle the additional capacity necessary for measuring, tracking and reporting on performance as required by the No Child Left Behind Act (NCLB).

Some have argued that increased assessment requirements contained in NCLB have forced teachers to "teach to the test," and have resulted in a less well-rounded curriculum. Recent instances of scoring errors, reporting delays and inaccurate student data have also been cited as evidence of too much testing. Others have argued annual assessments, as required in NCLB, play a critical role in improving student achievement by providing consistent information about school performance to inform decisions by educators, policymakers and parents. Regarding concerns about a narrowed curriculum resulting from NCLB accountability requirements, many say that if test results show that some schools need additional focus in the curriculum to help their students reach proficiency on core academic subjects, then doing so is time and effort well spent.

The Commission on No Child Left Behind recognizes the complexity and importance of these and other assessment and data issues. Witnesses at the hearing in Connecticut expressed very strong opinions regarding these and other issues. To view a complete webcast of the proceedings, click here. You can also access complete testimonies of all the witnesses by clicking here. As it develops recommendations on these issues for the upcoming reauthorization of NCLB, the Commission will consider information from a variety of sources including testimony from witnesses and others during our series of public hearings, written comments sent to the Commission, quality research and state, district and school performance data.

National Snapshot of Assessment and Data Systems

Testing
On a national level, student achievement, as measured by performance against state standards, appears to be improving. An interim report, National Assessment of Title I by the Independent Review Panel (IRP), released in February 2006, found that in states where three-year-trend data was available from 2000-2003, the percentage of students achieving at or above the state's proficient level rose for most student subgroups in a majority of states, although increases were often small.

The Center on Education Policy (CEP) also found that 78 percent of school districts surveyed for its study (From the Capital to the Classroom: Year 4 of the No Child Left Behind Act, released March 2006) reported that student achievement improved from 2003-2005 on the state tests used for NCLB. CEP also found statewide gains: 35 states said achievement had improved in reading during this time, and 36 states said it had improved in math. As Secretary Tommy Thompson, Co-Chairman of the Commission, noted in his opening remarks, "NCLB has brought a stronger focus on accountability for results, a commitment to assuring that all children—regardless of race or economic status—achieve at a high level."

The IRP reported, however, that based on available data from 20 states, most would not meet the goal of 100 percent proficiency by the 2013-2014 school year (as required under NCLB) unless the percentage of students achieving at the proficient level increased at a significantly faster rate.

In terms of implementation, the IRP found that as of March 2005, 27 states had fully implemented NCLB-required reading assessments, 26 states had fully implemented required mathematics assessments and 22 states had completed their first administration of required science assessments. While most states have met the requirement to annually assess 95 percent or more of their students, 14 states did not meet the minimum test participation requirement for one or more student subgroups.

Some Trends
Since the passage of NCLB, the number of states that report student achievement data has more than doubled. The IRP's National Assessment of Title I found that all 50 states now present data on their state report cards disaggregated (or broken down) by race/ethnicity and gender, as well as for limited English proficient (LEP) students, students with disabilities and low-income students. CEP's study reported that the vast majority of state and district officials believe NCLB's focus on the academic performance of student subgroups, particularly those that have traditionally lagged behind, is having a positive effect in schools.

A 2006 report released by the Education Sector (Margins of Error: The Education Testing Industry in the No Child Left Behind Era, Thomas Toch), however, found that 35 percent of states surveyed reported problems of data accuracy, while 20 percent failed to receive test results from a contractor in a timely manner. The report also cites industry assertions that testing companies often spend weeks tracking down test booklets that school systems have failed to forward, resolving discrepancies between enrollment figures and the number of students' tests, and cleaning up basic student biographical information and other factors required to accurately report test results under NCLB.

According to the Data Quality Campaign, no state data system currently includes every one of the 10 essential elements the Campaign defines as critical to longitudinal (or long-term) data systems. The question then becomes one of quality and utility: Are we able to effectively use the information gained from these annual assessments? Governor Roy Barnes, Co-Chairman of the Commission, raised this concern in his opening remarks, stating, "It makes little sense to assess our children if we can't accurately and effectively manage the data which is produced from these assessments. Effective data systems can and should be one of the most important priorities for a state."

What Does NCLB Currently Require?

Assessments
NCLB requires states to implement annual assessments, aligned with state standards, in reading and mathematics for grades 3-8 and at least once in grades 10-12 (seven different grade levels). Beginning in the 2007-2008 school year, states must also assess students in science once in each of three grade spans: 3-5, 6-9 and 10-12. States are required to provide for participation of all students including those with disabilities and LEP students. These requirements build on the 1994 authorization of the Elementary and Secondary Education Act, which required states to put in place standards and tests in reading and mathematics at three grade levels.

Results of required assessments must be reported for all subgroups of students including major racial/ethnic groups and low-income students, as well as those with disabilities and limited English proficiency. These results, along with other indicators such as graduation and attendance rates, form the basis of NCLB's accountability measures for schools and are used to identify schools for improvement, corrective action and restructuring, and interventions such as public school choice and supplemental education services (free tutoring).

Data Systems
Key to accurate use of student achievement results is a valid and reliable education data system. Before the passage of NCLB, few states had comprehensive data systems that reported data by subgroups of students. In addition, only a very small handful of states had data systems capable of tracking student achievement individually.

NCLB has very few requirements for the quality and structure of state data systems. The lack of statutory requirements, along with other factors, has led to large disparities in the quality, utility and validity of data systems among the states.

Some Observations From Witnesses

The panelists expressed varied perspectives on NCLB's annual testing provisions at the Commission's second hearing, held in a state that is no stranger to this issue. Betty Sternberg, Commissioner of the Connecticut State Department of Education, testified that the "massive, costly testing" required under NCLB provides no information to Connecticut educators that they did not already have under the state's pre-NCLB testing system. Rather, Sternberg asserted that annual testing costs money, time and effort, and does little to close the achievement gap. She asserted that "less large-scale, summative testing as currently required by NCLB, but better and more frequent formative testing—testing that works to improve student achievement, not just record it—would allow us to better focus on closing the unacceptably large differences in achievement among subgroups of children."

Commenting on Connecticut's suit against the U.S. Department of Education (see sidebar, "Conflict in Connecticut"), Attorney General Richard Blumenthal added that the "federal government is failing to fulfill its promise to children." According to Blumenthal, while NCLB requires annual assessments, the federal government has not provided states with adequate funding to implement this provision. This year alone, he asserted, Connecticut will be shortchanged $8 million in federal funds for assessment implementation. Blumenthal urged the federal government to "match their promise with resources."

William Taylor, Chair of the Citizens' Commission on Civil Rights, and Counsel to the NAACP in Connecticut v. Spellings, responded that in the 50 years he has worked on education and civil rights cases, he has seen too many instances where the focus of the case moves away from the interests of the child and instead devolves into a squabble over who should pay the bill. Taylor warned of this potential danger in Connecticut v. Spellings, a suit he claimed attacks the very "heart of the effort to improve educational opportunity for the most neglected children in America."

James Peyser, Chairman of the Massachusetts State Board of Education, offered a very different view of the value and cost of the annual assessments required by NCLB. Like Connecticut, Massachusetts had an assessment system in place before NCLB, in which students were tested in grades 4, 8 and 10 in all core academic subjects using a "healthy mix" of multiple-choice and open-response questions. Unlike Connecticut, however, Massachusetts views the increased testing required under NCLB as a "welcome addition" to its assessment system. "Annual testing is absolutely essential," Peyser said, because consistent information about student achievement over time allows educators and policymakers to make better informed judgments. Peyser added that, based on his state's experience, he could not in good faith argue that the amount of money states have received from the federal government to implement the additional tests has been inadequate.

New York City has also embraced NCLB's annual assessment requirements. "Testing is not a bad thing," said Joel Klein, Chancellor of the New York City Department of Education. "I want people to see tests as tools for us to find out whether teaching is working and whether what's taught is getting through to students." In addition to the statewide tests for grades 3-8, New York City will be launching an accountability initiative that uses in-class interim assessments as well as a value-added approach to measuring student achievement. "NCLB and annual testing are driving accountability and equity in a way that they have never been driven—we can't let go of that," Klein added.

In response to questions on whether NCLB's assessment requirements in reading and mathematics were taking time away from other subjects such as art, music and physical education, Klein stated that the critical issue is that too many children can't read. He said that no matter how many other wonderful subjects we offer our students, we are failing those who never learn how to read proficiently. Peyser added that in Massachusetts some schools are in fact choosing to reduce time spent on other subjects to focus on core academics. He said he believed these school leaders had made the right choice for their students because these are the core skills children need to succeed and they serve as the gateway to learning in other subjects.

While disagreement pervaded the assessment question, panelists did agree on the value of good data systems and the need for significant improvement in most states. Aimee Guidera, Director of the Data Quality Campaign, emphasized the importance of supporting state efforts to develop longitudinal data systems that track individual student performance from pre-K through 12th grade, even into postsecondary education. Currently, "we are data rich, but information poor," Guidera explained; longitudinal data systems will help educators evaluate and tailor their programs and make better informed decisions. Guidera further urged the federal government to build the capacity of all education stakeholders to use data: "Educators are scared of data; data has been seen as a hammer. We want them to see it as a flashlight, as the most important tool in their arsenal."

Stuart Kahl, President and CEO of Measured Progress, assured the Commission that, despite some struggles, testing companies have the capacity to handle the increased assessment demands of NCLB. However, Kahl asked that steps be taken to improve student information data systems because, "typically, the data files provided to the [testing] contractors are incomplete and contain inaccurate information. As any experienced researcher or testing person knows, far more time is spent cleaning data files than running final analyses." Kahl urged more time to allow companies to verify that each school had results for all its students, that subgroups are accurate and that no irregularities exist in students' response data.

Some Witnesses' Recommendations for Improving NCLB

• Develop an extensive pool of publicly available test items aligned with National Assessment of Educational Progress (NAEP) standards that could be used by schools and districts in creating interim assessments
• Reallocate existing funding to support the development of state data systems and the deployment of locally based assessment systems
• Support state efforts to develop accurate longitudinal data systems through incentives, directives or funding
• Promote the use of data for accountability and as a powerful tool for influencing classroom instruction
• Through ongoing professional development, build the capacity of every stakeholder in the education system to better use data to influence instruction and decision making
• Allow growth models to be applied in determining Adequate Yearly Progress (AYP) results

Using Data to Improve Student Achievement: Webster Hill Elementary School

After the hearing, commissioners visited Webster Hill Elementary School in West Hartford, CT. Webster tracks individual students from year to year and uses data from state, district and school assessments to set realistic goals and objectives for each of its students. Teachers receive ongoing training and professional development on how to interpret and use this data effectively in their classrooms. The school's principal and teachers said that this system allows them to better tailor lessons to their students' needs.

As for the annual testing requirements of NCLB, several Webster teachers explained that they agree with the notion that children need to be tested because a great deal can be discerned from such data. Some expressed concerns with the federal law, however, when it comes to testing some special education students at a grade level above that at which the students are currently performing. Instead, some urged the use of assessments that examine the individual student's needs and growth from year to year.

"We have year after year broken down student data. No Child Left Behind has helped us take the time to further look at the data. We're finding new ways to reach those hard-to-reach students."
—Kara Watson, 4th grade teacher, Webster Hill Elementary School

For Further Discussion
Assessments and quality data systems are critical to improving student achievement and closing the achievement gap in public schools. Although all 50 states now report disaggregated student information on their state report cards, participants at this hearing questioned the quality and utility of such data. Participants also expressed concerns over test format and frequency, as well as assessment requirements for special education and LEP students.

The Commission is seeking further comments from interested individuals on these topics:

• What successful strategies has your state or community used to implement the annual assessment provisions of NCLB?
• How have NCLB assessment requirements supported or hindered student achievement in your state or community?
• How has your state or community responded to the required assessments of special education students and LEP students?
• How can NCLB help to ensure that states, districts and schools develop and effectively use comprehensive data systems?

Comments can be submitted to the Commission via email at nclbfeedback@aspeninstitute.org

The Commission on No Child Left Behind is funded by the Bill & Melinda Gates Foundation, the Ewing Marion Kauffman Foundation, the Joyce Foundation, the John D. and Catherine T. MacArthur Foundation, Carnegie Corporation of New York and the Spencer Foundation. This document is published to communicate the results of the Commission's work. The findings, interpretations, and conclusions expressed in the Commission's documents are entirely those of the author(s) and should not be attributed in any manner to the donors.