¾«¶«Ó°Òµ

In this report we present a measure to identify unlikely patterns of correct/incorrect answers to test questions (commonly referred to as items). Some examples of why such patterns may occur include the misinterpretation of questions, item preknowledge, answer copying, or guessing behavior. The proposed measure is the probability of exceedance (PE). PE provides information about the probability of a correct/incorrect answer pattern, conditional on the test taker’s total score. Although this concept is not new, it is hardly if ever applied in practice.

When a test taker has prior knowledge about an administered test question (item), then this event is called item preknowledge, the test taker is called aberrant, and the item is called compromised. Item preknowledge negatively affects the corresponding testing program and its test score users (universities, companies, government organizations) because the scores produced for aberrant test takers will be invalid. The performance of eight statistics for detection of item preknowledge (five existing, two modified, and one new) was studied via computer simulations.

In the analysis of data for the Law School Admission Test (LSAT) and other similar standardized tests, a mathematical model called item response theory (IRT) is commonly used to estimate both the characteristics of the test questions (items) and the ability level of the test takers. Such analyses are based on the test takers’ correct and incorrect responses to the test items.