Sorry, you need to enable JavaScript to visit this website.

osse

Office of the State Superintendent of Education
 

DC Agency Top Menu

-A +A
Bookmark and Share

OSSE Completes 2013 DC CAS Test Integrity Investigations

Tuesday, March 18, 2014
Results highlight continued adherence to test integrity and security procedures

FOR IMMEDIATE RELEASE: March 18, 2014

CONTACT:  Ayan Islam; [email protected] (202) 316-1841

(WASHINGTON, DC) – Today, the Office of the State Superintendent of Education (OSSE) released the results of its annual independent investigation into test integrity and security procedures for the 2013 District of Columbia Comprehensive Assessment System (DC CAS).  The results of OSSE's test integrity investigation revealed no systemic issues with the implementation of DC CAS, and less than 0.2% of the total testing groups with violations in testing procedures critical enough to invalidate scores in those groups.

The statewide DC CAS exam was administered on April 22-May 3, 2013, at 195 schools within 52 Local Education Agencies (LEAs), including DC Public Schools (DCPS) and 51 public charter LEAs, and was taken by 32,838 students.

OSSE's test integrity procedures include selecting testing groups for investigation using a sophisticated flagging methodology, which analyzes statistical significance in six areas of test results.  Additionally, several test groups were flagged for investigation through a random selection process. An independent contractor, Alvarez and Marsal, LLC (A&M), then conducted on-site investigations, including data and document review and interviews with school administrators, teachers, and students.

Of the 2,032 testing groups in 195 schools during the 2013 DC CAS implementation, a total of 45 testing groups* in 37 schools, representing 17 LEAs were selected for investigation. Of the 45 testing groups flagged for investigation, less than 0.2%, or four (4) were deemed to have “Critical” findings (test tampering or academic fraud); six (6) were found to have “Moderate” findings (defined violations but not test tampering or academic fraud); and seven (7) were found to have “Minor” findings (test administration errors). The remaining testing groups had no findings (one school identified for test integrity investigations was closed at the end of the 2012-2013 school year so no investigation was conducted).

“The latest results show that the overwhelming majority of District educators are committed to maintaining the integrity and security of our DC CAS process for our students,” said State Superintendent Jesús Aguirre. “Any intent to thwart the DC CAS process obviously undermines the hard work of our educators, and deprives our students of the opportunities to receive targeted services, so we are pleased that, again, the integrity of our assessment was validated and remains a reliable tool as we work to improve instruction for our students.”

For the third consecutive year, Alvarez and Marsal, LLC (A&M) was contracted to conduct test integrity investigations on behalf of OSSE.  Testing groups in District of Columbia Public Schools (DCPS) and District Public Charter Schools (PCS) were identified for possible testing irregularities after an internal analysis of the 2013 DC CAS Reading and Mathematics results. 

The 2013 DC CAS results for the four (4) testing groups, one in each of four different schools, found to have “Critical” test security violations will be invalidated, and the school will be required to create a corrective action plan and will be subject to enhanced monitoring by OSSE during the 2014 DC CAS testing period. Other consequences for the schools found to have had moderate and minor determinations, include letters of reprimand and corrective action plans.  LEAs may also elect to enforce further accountability measures in their schools with test integrity violations.

Background

The testing groups chosen for review, as part of OSSE’s test integrity process, were selected using random classroom sampling (newly required by the Testing Integrity Act of 2013), and forensic analyses of six (6) flags:

  • wrong-to-right erasures;
  • person-fit analysis (unusual response patterns);
  • score growth;
  • score drops;
  • performance variations between multiple choice and constructed response; and
  • difference in performance on previously unused items. 

A testing group is selected for integrity investigation when any, or a combination, of the following occurs:

  • testing groups trigger two or more flags;
  • there is an extreme value in a single flag;
  • consecutive years of wrong-to-right erasures;
  • the same flag occurred across multiple subjects; or
  • inclusion in random sampling. 

The score drop analysis in the flagging criteria began last year and helped to further detect testing irregularities. 

Applying the methodology to the test results from the 2013 DC CAS, OSSE flagged testing groups in both DCPS and public charter schools for further review.

Each year, during the review of the DC CAS administration, OSSE analyzes and enhances its flagging methodology with the objective of ensuring the validity of the data used for calculating metrics in the District’s accountability system. The enhancements for the 2013 testing year was a result of an extensive vetting process with OSSE’s Technical Assistance Committee, comprised of national assessment and psychometrics experts, as well as requirements by the recently enacted Testing Integrity Act of 2013. 

Moreover, for the 2013-2014 DC CAS administration, LEAs will submit signed affidavits from test administrators from OSSE within 15 days after the DC CAS testing window and testing and security agreements will be distributed requiring the signatory to acknowledge that he/she understands that knowingly and willingly violating the law, regulation and/or plans could result in the loss of an OSSE certification and license.

SY 2012-2013 DC CAS Test Integrity Results

SY 2012-2013 DC CAS Test Integrity Results Presentation