Search

Purification of the test has been a well-accepted procedure in enhancing the performance of tests for differential item functioning (DIF). As defined by Lord, purification requires reestimation of ability parameters after removing DIF items before conducting the final DIF analysis. IRTPRO 3 is a recently updated program for analyses in item response theory, with built-in DIF tests but not purification procedures. A simulation study was conducted to investigate the effect of two new methods of purification. The results suggested that one of the purification procedures showed significantly improved power and Type I error. The procedure, which can be cumbersome by hand, can be easily applied by practitioners by using the web-based program developed for this study.

This study established an effect size measure for differential functioning for items and tests' noncompensatory differential item functioning (NCDIF). The Mantel-Haenszel parameter served as the benchmark for developing NCDIF's effect size measure for reporting moderate and large differential item functioning in test items. The effect size of NCDIF is influenced by the model, the discrimination parameter, and the difficulty parameter. Therefore, tables of NCDIF's effect size were presented at given levels of "a," "b," and "c" parameters. In addition, a general effect size recommendation for moderate and large NCDIF is also established.

Raju, van der Linden, and Fleer (1995) introduced a framework for differential functioning of items and tests (DFIT) for unidimensional dichotomous models. Since then, DFIT has been shown to be a quite versatile framework as it can handle polytomous as well as multidimensional models both at the item and test levels. However, DFIT is still limited by the two-group comparison (reference vs. focal group). In real testing situations, it is quite common that a practitioner or researcher may want to compare more than two groups simultaneously. Ethnicity, for example, may have three or more categories. This current study extends the two-group noncompensatory differential item functioning index (NCDIF) in DFIT to multiple-group NCDIF. The significance test developed for the two-group NCDIF and effect size were utilized in the multiple-group cases. The effectiveness of this new approach is demonstrated in a simulation study where various degrees and types of DIF are embedded.

In a typical differential item functioning (DIF) analysis, a significance test is conducted for each item. As a test consists of multiple items, such multiple testing may increase the possibility of making a Type I error at least once. The goal of this study was to investigate how to control a Type I error rate and power using adjustment procedures for multiple testing, which have been widely used in applied statistics. In the simulation, four distinct DIF methods were performed under various testing conditions. The methods were the Mantel-Haenszel (MH) method, the logistic regression (LR) procedure, the Differential Functioning Item and Test (DFIT) framework, and Lord's chi-square test. As an adjustment procedure, the Bonferroni correction, Holm's procedure, or the Benjamini and Hochberg (BH) false discovery rate was applied. The results showed the MH and the LR clearly benefited from Holm's and the BH adjustments, whereas the DFIT and Lord's chi-square test did not require adjustments for conditions under this study. (Contains 2 tables.)

Oshima, Raju, and Flowers demonstrated the use of an item response theory-based technique for analyzing differential item function (DIF) and differential test function for dichotomously scored data that are intended to be multidimensional. Their study assumed that the number of intended-to-be measured dimensions was correctly identified. In practice, however, the number of dimensions may be misidentified. Therefore, the purpose of this study was to demonstrate the effects of both underestimation and overestimation of the number of intended-to-be measured dimensions on the multidimensional DIF analysis using simulated two-dimensional data with known DIF items. Results show that overestimation of the number of y traits had a consequence of decreased power. Underestimation resulted in missing a certain type of nonuniform DIF, as well as confounding the impact with DIF. Recommendations are made on how to conduct a DIF investigation with a multidimensional within-item test. (Contains 2 figures and 5 tables.)

Nambury S. Raju (1937-2005) developed two model-based indices for differential item functioning (DIF) during his prolific career in psychometrics. Both methods, Raju's area measures (Raju, 1988) and Raju's DFIT (Raju, van der Linden, & Fleer, 1995), are based on quantifying the gap between item characteristic functions (ICFs). This approach provides an intuitive and flexible methodology for assessing DIF. The purpose of this tutorial is to explain DFIT and show how this methodology can be utilized in a variety of DIF applications.

Oshima, Raju, Flowers, and Slinde (1998) described procedures for identifying sources of differential functioning for dichotomous data using differential bundle functioning (DBF) derived from the differential functioning of items and test (DFIT) framework (Raju, van der Linden, & Fleer, 1995). The purpose of this study was to extend the procedures for dichotomous DBF to the polytomous case and to illustrate how DBF analysis can be conducted with polytomous scoring, common to psychological and educational rating scales. The data set used was parent and teacher ratings of child problem behaviors. Three group contrasts (teacher vs. parent, boy vs. girl, and random groups) and two bundle organizing principles (subscale designation and random selection) were used for the DBF analysis. Interpretations of bundle indexes in the context of child problem behaviors were presented.

An examinee-level (or conditional) reliability is proposed for use in both classical test theory (CTT) and item response theory (IRT). The well-known group-level reliability is shown to be the average of conditional reliabilities of examinees in a group or a population. This relationship is similar to the known relationship between the square of the conditional standard error of measurement ("SEM") and the square of the group-level "SEM". The proposed conditional reliability is illustrated with an empirical data set in the CTT and IRT frameworks. (Contains 3 tables.)

A new item parameter replication method is proposed for assessing the statistical significance of the noncompensatory differential item functioning (NCDIF) index associated with the differential functioning of items and tests framework. In this new method, a cutoff score for each item is determined by obtaining a (1-alpha ) percentile rank score from a frequency distribution of NCDIF values under the no-DIF condition by generating a large number of item parameters based on the item parameter estimates and their variance-covariance structures from a computer program such as BIILOG-MG3. This cutoff for each item can be used as the basis for determining whether a given NCDIF index is significantly different from zero. This new method has definite advantages over the current method and yields cutoff values that are tailored to a particular data set and a particular item. A Monte Carlo assessment of this new method is presented and discussed.

This note describes a procedure for estimating the range restriction component used in correcting correlations for unreliability and range restriction when an estimate of the reliability of a predictor is not readily available for the unrestricted sample. This procedure is illustrated with a few examples. (Contains 1 table.)

Much interest exists among parents and researchers regarding the benefits and drawbacks of delaying kindergarten entrance to acquire academic advantage ("redshirting"). How evident is this assumed advantage at the kindergarten level and beyond? The authors evaluated large-scale test data from Grades K-8 to investigate the difference in performance between younger children (summer birthday) and older children (fall birthday). The performance gap evident in kindergarten decreased rapidly in Grades 1-3 but persisted up to Grade 5, until leveling off at middle school. The performance gap in the early grades that resulted from birth date was much larger than was the gap caused by gender difference. (Contains 2 tables and 2 figures.)

Examined the polytomous differential functioning of items and tests (DFIT) framework proposed by N. Raju and others through simulation. Findings show that the DFIT framework is effective in identifying differential item functioning and differential test functioning. (SLD)

A general framework for assessing differential functioning of items and tests (DFIT) that was recently proposed is expanded to include differential bundle functioning as a mechanism for identifying sources of differential functioning. The expansion is illustrated with a dataset of 4000 fourth-grade students from a reading comprehension examination. (SLD)

Estimated Type I error rates for three tests that compare means by testing data from two independent samples: (1) the independent samples "t" test; (2) B. Welch's approximate degrees of freedom test (1938); and (3) G. James's second-order test (1951, 1954). Results provide guidance about the total sample sizes required for controlling Type I error rates. (SLD)

How item bias indexes based on item response theory (IRT) identify bias that results from multidimensionality is demonstrated. Simulation results suggest that IRT-based bias indexes detect multidimensional items with bias but do not detect multidimensional items without bias. They also do not confound between-group differences on the primary test. (SLD)

The recent study of Oshima, Raju, and Nanda proposes the item parameter replication (IPR) method for assessing statistical significance of the noncompensatory differential item functioning (NCDIF) index within the differential functioning of items and tests (DFIT) framework. Previous Monte Carlo simulations have found that the appropriate cutoff values for determining statistical significance of NCDIF depend on sample size and the item response theory (IRT) model used for the analysis. The IPR method simplifies the process of identifying cutoff values that are tailored to a particular research setting. This approach has been shown to be effective for detecting differential item functioning (DIF) in dichotomous items. The current article extends the IPR method to the polytomous case. [ABSTRACT FROM AUTHOR]