Investigating Item Stability: An Empirical Investigation into the Variability of Item Statistics Under Conditions of Varying Sample Design and Sample Size. Occasional Paper No. 18.

The stability of Rasch test item difficulty parameters was investigated under varying conditions. Data were taken from a mathematics achievement test administered to over 2,000 Australian students. The experiments included: (1) relative stability of the Rasch, traditional, and z-item difficulty parameters using different sample sizes and designs; (2) effect of different sample types and sizes on the Rasch item fit estimator; (3) effects, on the item fit parameter and the Rasch item difficulty parameter, of removing some less appropriate items from the test; and (4) an examination of Wright's statement that the standard error of the item difficulty parameter is a good estimator of its variance, and that it has an inverse square root relationship to the sample size. Results showed that Rasch and z-item difficulty parameters were similar. Item fit increased as sample size increased. The removal of poorly fitting items improved fit values for the entire test, but worsened them for the remaining items. The Rasch standard error parameter was an appropriate measure of the true error of estimation as calculated from the square root of the sampling variance of the item difficulty index. (Implications for test calibration are concluded, and detailed item analyses are appended.) (GDC)