Search

Current interest in the assessment of measurement equivalence emphasizes two methods of analysis, linear, and nonlinear procedures. This study simulated data using the graded response model to examine the performance of linear (confirmatory factor analysis or CFA) and nonlinear (item-response-theory-based differential item function or IRT-Based DIF) methods. Two CFA procedures, lambda (slope) structure only and lambda and tau (slope and intercept) structures, were used to examine measurement equivalence across focal and reference groups. An IRT-based, noncompensatory DIF (NC-DIF) procedure (N. Raju, W. van der Linden, and P. Fleer, 1995) was also used to examine measurement equivalence across groups. Results indicate that the lambda procedure successfully identified items that had differences in the alpha-parameters, but did not identify items that had differences in the beta-parameters. The lambda/tau and NC-DIF procedures identified items that had differences in the beta-parameters. The lambda-tau and NC-DIF procedures were not, however, as sensitive to items that had differences only in the alpha-parameters. When the focal and reference groups had different ability distributions (or impact), the lambda/tau procedure had a lower (or an acceptable) Type II error rate (in detecting true positives), but had a much higher (or an unacceptable) Type I error rate (in detecting false positives). The NC-DIF procedure appeared to have acceptable Type I and Type II error rates in both no-impact and impact scenarios. (Contains 5 tables and 23 references.) (Author/SLD)

This study was patterned after a previous study by Skaggs and Lissitz (1992) in which inconsistency of differential item functioning (DIF) was reported across test administrations. They suggested multidimensionality of test data as one possible reason for inconsistency. Therefore, in this study, DIF indices which were developed recently with a multidimensional perspective were included. In addition, the consistency of differential test functioning (DTF) was evaluated. DIF/DTF analyses were conducted for both gender and ethnic differences. Ten random samples of 1,000 examinees from each gender and ethnic category were taken from a math basic skills test which was administered in a statewide testing program in two separate years (1984 and 1987). In general, results indicated a more favorable evaluation of the consistency of DIF indices than the Skaggs and Lissitz study. Possible reasons for conflicting conclusions are discussed. (Contains 18 references and 9 tables.) (Author/SLD)

This paper evaluated multidimensional linking procedures with which multidimensional test data from two separate calibrations were put on a common scale. Data were simulated with known ability distributions varying on two factors which made linking necessary: mean vector differences and variance-covariance (v-c) matrix differences. After the calibrations of multidimensional item parameters, blocks of means from item parameter estimates were used to equate the two groups. The linking was effective for mean vector differences. The linking for v-c matrix differences was less effective, but encouraging. Suggestions for future research are provided. Four tables are attached. (Contains 6 references.) (Author)

The purpose of this study was to introduce a procedure to detect differential item functioning (DIF) particularly suitable for criterion-referenced tests and to demonstrate how this approach would affect the identification of DIF items using real data sets. The procedure based on item response theory (IRT) assesses DIF at a limited closed interval of thetas at which a cutoff score fails. Illustrative data showed that identification of DIF could be quite different with this unconventional procedure as opposed to traditional DIF measures with which DIF was assessed over the entire range of ability. It was recommended that test development practitioners be actively involved in the DIF analysis to investigate not only if an item is biased, but where it may be biased. Three figures, and five tables are included. (Contains 8 references.) (Author)

In 1992 N. S. Raju and others proposed a general procedure for assessing differential test functioning (DTF) and item bias (differential item functioning or DIF) in tests developed with unidimensional, multidimensional, or polytomous item response theory (IRT) models. The purpose of this paper is to assess the adequacy and validity of their technique in the two-dimensional IRT setup. Following Raju and others, a chi square test is described for determining whether or not an observed DTF is significantly different from zero. When an observed DTF is statistically significant, one may begin the search for items that may be causing the significant chi square. After deletion of items with a high positive compensatory DIF, the DTF index and its chi square should be recomputed. The technique is illustrated with generated 40-item two-dimensional data sets. Once the multidimensional item parameters are given and differences in item parameters between the reference and focal groups are found, the new procedure identifies DTF and DIF as expected. Five tables present analysis results. (Contains 15 references.) (SLD)

The effect of violating the assumption of nonspeededness on ability and item parameter estimates in item response theory was studied through simulation under three speededness conditions. Results indicate that ability estimation was least affected by speededness but that substantial effects on item parameter estimates were found. (SLD)

A two-stage procedure for estimating item bias was examined with six indices of item bias and the Mantel-Haenszel statistic. Results suggest that the two-stage procedure is not very useful when the number of biased items is small and bias magnitude is weak. (SLD)