Each year, the Office of the State Superintendent of Education (OSSE) reviews statewide assessment administration to ensure the validity and reliability of the assessment results. As part of this process, OSSE reviews test security protocols and data to ensure the tests were administered with fidelity. OSSE takes breaches of test security very seriously, as violations of any nature can jeopardize confidence in the accuracy of our assessment results. At the same time, it is important to note that violations are intended to capture a broad range of issues in test administration and do not necessarily indicate wrongdoing or intentional misconduct.
For the 2014-15 and 2015-16 administration years, OSSE partnered with Caveon Test Security, a nationally recognized firm with substantial experience in conducting test integrity investigations. Caveon conducted data forensics analysis of test response data to identify anomalous results, and conducted post-administration investigations to attempt to understand what may have caused statistical anomalies identified in the test response data. The methodology used to flag schools included considering similarities in individual students’ response patterns; looking at levels of student response changes (commonly known as “erasure analysis” or “response change analysis”); and looking at unusual changes in scores, year to year. The onsite post-administration investigation included site visits, document reviews, and one-on-one interviews.
Test integrity violations are classified in the following manner:
- No findings: No evidence of test security violations, wrongdoing, or error
- Minor: Minor misadministration errors (e.g., incomplete or missing test documents, inconsistencies in the application of administration procedures)
- Moderate: Moderate misadministration errors (e.g., misinterpretation of requirements in the Test Coordinator/Administrator Manuals not related to test tampering or academic fraud; failure to report certain types of incidents; failure to distribute, collect, or refusal to sign Non-Disclosure Agreements)
- Significant: Substantiated isolated test security violations; test tampering or academic fraud (e.g., educator coaching; providing students with answers; allowing student use of calculators or other technology when prohibited)
- Critical1: Substantiated systemic test security violations; school- or LEA-led test tampering or academic fraud (e.g., school- or LEA-level coordination of educator coaching or test tampering; school-wide test fraud)
Please see below the results of the statewide test integrity investigations:
2015-16 Test Integrity Investigation Reports - coming soon
2014-15 Test Integrity Investigation Reports - coming soon
2013-14 Test Integrity Investigation Reports
2012-13 Test Integrity Investigation Reports
2011-12 Test Integrity Investigation Reports
2010-11 Test Integrity Investigation Reports