A Comparative Study of Observed Score Approaches and Purification Procedures for Detecting Differential Item Functioning.

The purposes of this study were to introduce the iterative purification procedure and to compare this with the two-step purification procedure, to compare false positive error rates and the power of five observed score approaches and to identify factors affecting power and false positive rates in each method. This study used 2,400 data sets that were divided into uniform, symmetric nonuniform, and nonsymmetric nonuniform differential item functioning (DIF) data sets. The sample size pairs were either 500,500 or 1,000,1,000 for the reference group and the focal group when the means of ability distributions for the 2 groups were the same, and either 1,000,500 or 1,000,250 for the reference and focal groups when the means of ability distributions for the 2 groups were different. Each dataset included four items with uniform, symmetric nonuniform, or nonsymmetric nonuniform DIF, with each DIF item having either a 0.4 or 0.8 amount of DIF (that is, the area between two item characteristic curves). The purification procedures reduced false positive error rates and/or increased power. The Mantel Haenszel method was superior to other methods with uniform DIF data sets, and the Absolute Mean Deviation method using the iterative purification procedure was superior to the others in nonuniform data sets when the means of ability distributions for the two groups were different. The ability estimation and the sample size affected detection rates and false positive error rates for all methods. The DIF effect size was also a strong influence on detection rates. (Contains 21 tables and 25 references.) (Author/SLD)