Search

Accurate parameter estimation in the Rasch model involves the assumption of conditional independence, also termed local independence. Conditional on ability, the responses to items A and B should be independent. Two types of conditional dependence are detailed in this pedagogical piece: trait dependency and response dependency. The bias in difficulty and reliability and the estimates of fit and correlated residuals resulting from these dependencies are compared and contrasted to results from using models that account for the dependency. Contrasts with results from a 2-parameter item response theory model are also briefly noted.

Using a scale of test-taking motivation designed to have multiple factors, results are compared from a confirmatory factor analysis (CFA) using LISREL and a multidimensional Rasch partial credit model using ConQuest. Both types of analyses work with latent factors and allow the comparison of nested models. CFA models most typically model a linear relationship between observed and latent variables, while Rasch models specify a non-linear relationship between observed and latent variables. The CFA software provides many more measures of overall fit than ConQuest, which is focused more on the fit of individual items. Despite the conceptual differences in these techniques, the results were similar. The data fit a three-dimensional model better than the one-dimensional or two-dimensional models also hypothesized, although some misfit remained.

The validity of inferences based on achievement test scores is dependent on the amount of effort that examinees put forth while taking the test. With low-stakes tests, for which this problem is particularly prevalent, there is a consequent need for psychometric models that can take into account differing levels of examinee effort. This article introduces the effort-moderated IRT model, which incorporates item response time into proficiency estimation and item parameter estimation. In two studies of the effort-moderated model when rapid guessing (i.e., reflecting low examinee effort) was present, one based on real data and the other on simulated data, the effort-moderated model performed better than the standard 3PL model. Specifically, it was found that the effort-moderated model (a) showed better model fit, (b) yielded more accurate item parameter estimates, (c) more accurately estimated test information, and (d) yielded proficiency estimates with higher convergent validity. (Contains 3 notes.)

Type I error rates for PARSCALE's fit statistic were examined. Data were generated to fit the partial credit or graded response model, with test lengths of 10 or 20 items. The ability distribution was simulated to be either normal or uniform. Type I error rates were inflated for the shorter test length and, for the graded-response model, also for the longer test length when the ability distribution was uniform. In conditions in which [alpha] was inflated, it was particularly high when one or more response categories were used infrequently. Overall, PARSCALE's fit index is not recommended for short tests.

Three methods of detecting item drift were compared: the procedure in BILOG-MG for estimating linear trends in item difficulty, the CUSUM procedure that Veerkamp and Glas (2000) used to detect trends in difficulty or discrimination, and a modification of Kim, Cohen, and Park's (1995) x 2 test for multiple-group differential item functioning (DIF), using linear contrasts on the discrimination and difficulty parameters. Data were simulated as if collected over 3, 4, or 5 time points, with parameter drift in either a gradual, linear pattern, a less linear but still monotonic pattern, or as a sudden shift at the third time point. The BILOG-MG procedure and the modification of the Kim et al. procedure were more powerful than the CUSUM procedure, nearly always detecting drift. All three procedures had false alarm rates for nondrift items near the nominal alpha. The procedures were also illustrated on a real data set.

The use of "neutral or unsure" on an instrument designed to measure identity development in college students was explored. The nominal response model from item response theory was used to evaluate whether "neutral or unsure" was used more frequently by those at low or middle levels of development; the results depended on the subscale and sometimes on the item within the subscale. Scoring based on the nominal response model allows for this category to be treated differently for different items.

Marginal maximum likelihood, a common estimation method for item response theory models, is not inherently a Bayesian procedure. However, due to estimation difficulties, Bayesian priors are often applied to the likelihood when estimating 3PL models, especially with small samples. Little focus has been placed on choosing the priors for marginal maximum estimation. In this study, using sample sizes of 1,000 or smaller, not using priors often led to extreme, implausible parameter estimates. Applying prior distributions to the c -parameters alleviated the estimation problems with samples of 500 or more; for the samples of 100, priors on both the a -parameters and c -parameters were needed. Estimates were biased when the mode of the prior did not match the true parameter value, but the degree of the bias did not depend on the strength of the prior unless it was extremely informative. The root mean squared error (RMSE) of the a -parameters and b -parameters did not depend greatly on either the mode or the strength of the prior unless it was extremely informative. The RMSE of the c -parameters, like the bias, depended on the mode of the prior for c. [ABSTRACT FROM AUTHOR]

The eigenvector could then be computed from Equation 3 without actually computing the eigenvalues: HT ht and HT ht . If desired, the researcher could normalize the M4 weights by dividing each weight by HT ht to make the variance of the composite equal one. [Extracted from the article]

Equivalently, one could use proportions in place of counts, and at the population level, HT ht . For DIF-free items, the population value of SB I i i sb would then be HT ht . Each HT ht is compared with the median HT ht to assess item-level group differences beyond any overall ability difference. [Extracted from the article]

In structural equation modeling software, either limited-information (bivariate proportions) or full-information item parameter estimation routines could be used for the 2-parameter item response theory (IRT) model. Limited-information methods assume the continuous variable underlying an item response is normally distributed. For skewed and platykurtic latent variable distributions, 3 methods were compared in Mplus: limited information, full information integrating over a normal distribution, and full information integrating over the known underlying distribution. Interfactor correlation estimates were similar for all 3 estimation methods. For the platykurtic distribution, estimation method made little difference for the item parameter estimates. When the latent variable was negatively skewed, for the most discriminating easy or difficult items, limited-information estimates of both parameters were considerably biased. Full-information estimates obtained by marginalizing over a normal distribution were somewhat biased. Full-information estimates obtained by integrating over the true latent distribution were essentially unbiased. For the a parameters, standard errors were larger for the limited-information estimates when the bias was positive but smaller when the bias was negative. For the d parameters, standard errors were larger for the limited-information estimates of the easiest, most discriminating items. Otherwise, they were generally similar for the limited- and full-information estimates. Sample size did not substantially impact the differences between the estimation methods; limited information did not gain an advantage for smaller samples. [ABSTRACT FROM AUTHOR]

The article discusses several BILOG software options in the context of separate calibration. It states that using two BILOG runs to keep the item parameters fixed with NOADJUST. It notes that the NOADJUST option is important in keeping anchor items fixed at their input values. It suggests practitioners to use BILOG when using fixed-anchor item scaling.

Four item response theory (IRT) models were compared using data from tests where multiple items were grouped into testlets focused on a common stimulus. In the bi-factor model each item was treated as a function of a primary trait plus a nuisance trait due to the testlet; in the testlet-effects model the slopes in the direction of the testlet traits were constrained within each testlet to be proportional to the slope in the direction of the primary trait; in the polytomous model the item scores were summed into a single score for each testlet; and in the independent-items model the testlet structure was ignored. Using the simulated data, reliability was overestimated somewhat by the independent-items model when the items were not independent within testlets. Under these nonindependent conditions, the independent-items model also yielded greater root mean square error (RMSE) for item difficulty and underestimated the item slopes. When the items within testlets were instead generated to be independent, the bi-factor model yielded somewhat higher RMSE in difficulty and slope. Similar differences between the models were illustrated with real data. [ABSTRACT FROM AUTHOR]

Type I error rates were examined for several fit indices available in GGUM2000: extensions of Infit, Outfit, Andrich's χ[Sup2] and the log-likelihood ratio χ[Sup2]. Infit and Outfit had Type I error rates much lower than nominal α. Andrich's χ[Sup2] had Type I error rates much higher than nominal α, particularly for shorter tests or larger sample sizes. The log-likelihood χ[Sup2] had Type I error rates near or below nominal α for small samples or longer tests but had inflated error rates with large samples and shorter tests. For conditions in which the log-likelihood ratio χ[Sup2] did not perform well, alternative fit indices or modifications to these procedures should be considered in future studies. [ABSTRACT FROM AUTHOR]

Investigates the factors that affect the recovery of item parameters in the nominal response model. Background on the nominal response model; Mathematical function of the model; Manipulation of the distribution of the examinee traits and item discrimination in the study.