FOR IMMEDIATE RELEASE: April 12, 2013
CONTACT: Athena W. Hernandez (OSSE) 202.724.1513; [email protected]
WASHINGTON, DC – The Office of the State Superintendent (OSSE) today released findings from the investigation into test integrity and security procedures for the 2012 District of Columbia Comprehensive Assessment System (DC CAS). The statewide DC CAS exam was administered on April 17-27, 2012, at 243 schools and taken by 32,624 students in both District of Columbia Public Schools (DCPS) and Public Charter Schools (PCS). OSSE commissioned Alvarez and Marsal, LLC (A&M) for a second consecutive year to investigate testing groups* in DCPS and PCS that were flagged for possible testing irregularities after an internal analysis of the 2012 DC CAS Reading and Mathematics test results.
Of the 2,688 testing groups in 243 schools across the District, a total of 41 testing groups in 25 schools were investigated by A&M. Furthermore, out of the 25 schools flagged for investigations, 11 schools were categorized as “Critical” (serious test security violations); four (4) schools as “Moderate” (anomalies with defined violations but not test tampering); and one (1) school as “Minor” (procedural infractions).The remaining nine schools were categorized as having no findings.
As a result of the investigations, in the eleven (11) schools found to have “Critical” test security violations, the 2012 DC CAS Mathematics and Reading test scores of the 18 testing groups will be invalidated out of the 41 investigations. Other consequences, for the moderate and minor determinations, include letters of reprimand and corrective action plans.
“OSSE remains committed to safeguarding the integrity of testing procedures and enforcing policies that ensure transparency in the process,” said State Superintendent Hosanna Mahaley Jones. “While it appears we solely focus on measuring student achievement through statewide testing, we equally concentrate on test security measures and accountability. This is integral to our mission as we need to accurately measure student achievement in order to support student success and improve school effectiveness. If any school is found in violation of testing policies and protocols, we will act swiftly and apply the necessary sanctions.”
To better inform the investigative process for the 2012 testing year, A&M enhanced the existing methodology and protocols to include “Student-Level Data.” This enhancement allowed investigators to more adequately identify testing groups and students flagged for investigation, and streamlined the interview process.
“Applying standardized processes involving site visits, document reviews, and one-on-one interviews, we have completed a thorough and uncompromising investigation on behalf of OSSE,” explained A&M Managing Director Joseph Gardemal. “We considered testing environments, interviewed administrators and students, and reviewed documents to substantiate our findings.”
The testing groups chosen for further review were selected using a forensic analysis of four (4) flagging mechanisms: wrong-to-right erasures, score variation within classrooms, unusual student gains from 2010 to 2012, and significant score drops. When testing groups are flagged in two (2) of the four (4) methods, or when consecutive years of wrong-to-right erasures occurs, it triggers the initiation of investigations by OSSE. The inclusion of the score drop analysis in the flagging criteria this year helped to further detect testing irregularities.
“The methods that OSSE uses cover multiple ways that testing irregularities can be manifest,” said William Schafer, DC CAS Technical Advisory Committee member and University of Maryland professor. “In the development of their approach and throughout its implementation, OSSE has kept its technical advisors informed of their procedures and results and has continually placed considerable resources in the use and study of their flagging criteria. As a result, OSSE has a credible and effective process for identification of schools for further investigation.”
“In the face of uncompromising evidence, OSSE has held firm to its protocols and standards and is demanding accountability for actionable offenses,” added A&M. “We stand with them in the decision to invalidate test scores in scenarios where test security measures were compromised.”
The administration of the tests included a strategic series of preventive, detection, and investigative measures, which included:
• Enhanced training for LEA test chairpersons
• 104 schools were monitored by OSSE during the testing window
• Testing “Violation Form” title was changed to “Incident Reports” to encourage communication between OSSE and the LEAs on testing irregularities
• Test security seals were placed on test booklets
• Material tracking procedures were implemented
“Each year we review and assess our policies to ensure there is sound test security,” added State Superintendent Mahaley Jones. “While this year’s investigation has yielded more testing groups with critical infractions in testing protocols, it also serves as proof that 99.4% are following the rules.”
A copy of the Test Integrity Power Point presentation, redacted copy of the findings report, and executive summary can be found on www.osse.dc.gov.
###
Editor Note:
* A testing group is defined as a classroom of students or multiple classrooms tested in one environment with one test administrator.
About Alvarez and Marsal, LLC
Commissioned in 2011 and 2012 to investigate classrooms, strengthen data integrity measures, ensure the agency’s testing methods and confirm accuracy of test results, Alvarez and Marsal, LLC created a multi-disciplined team of professionals from the Public Sector Services (PSS) and Global Forensic and Dispute Services (GFD) practices with backgrounds in investigations, independent assessments, and experience working on behalf of K-12 education clients.
Joining the practices culminated in a team with deep investigative capabilities and a track record of working on behalf of school districts. A&M’s heritage in fact-based performance drives a commitment to identifying deviations from established policies as testing was implemented at the classroom level. The team from the A&M Washington, D.C. office worked on the investigations.
For more information about OSSE, go to: www.osse.dc.gov; www.twitter.com/OSSEDC; www.facebook.com/OSSEDC
