Search

Simulation studies are needed to investigate how many score categories are sufficient to treat ordered categorical data as continuous, particularly for bifactor models. The current simulation study aims to address such needs by investigating the performance of estimation methods in the bifactor models with ordered categorical data. Results support the application of categorical estimators to the ordered categorical data rather than the continuous estimators when sample size is large (750). Otherwise, an applied researcher may have to use the continuous estimators due to the model non-convergence. In this circumstance, the number of response categories needs to be at least 6 to avoid the rejection of correctly specified bifactor models by the chi-square test and estimate the model parameters accurately. The robust maximum likelihood (MLR) may be chosen among two continuous estimators due to its smaller type I error rate for the chi-square test than the ML. Practical implications of study findings are discussed. [ABSTRACT FROM AUTHOR]

Validity of performance standards is a key element for the defensibility of standard setting results, and validating performance standards requires collecting multiple pieces of evidence at every step during the standard setting process. This study employs a statistical procedure, latent class analysis, to set performance standards and compares latent class analysis results with previously established performance standards via the modified‐Angoff method for cross‐validation. The context of the study is an operational large‐scale science assessment administered in one of the southern states in the United States. Results show that the number of classes that emerged in the latent class analysis concurs with the number of existing performance levels. In addition, there is a substantial level of agreement between latent class analysis results and modified‐Angoff method in terms of classifying students into the same performance levels. Overall, the findings establish evidence for the validity of the performance standards identified via the modified‐Angoff method. Practical implications of the study findings are discussed. [ABSTRACT FROM AUTHOR]

This study employs the 4‐parameter logistic item response theory model to account for the unexpected incorrect responses or slipping effects observed in a large‐scale Algebra 1 End‐of‐Course assessment, including several innovative item formats. It investigates whether modeling the misfit at the upper asymptote has any practical impact on the parameter estimates. With a simulation study, it also investigates the amount of bias in the parameter estimates when the slipping effects are ignored. Findings from the empirical data indicate that the impact of ignoring slipping effects is negligible when the abilities are evaluated within the context of classification of students into performance levels; however, it is present toward the extreme ends of ability continuum within the context of individual abilities. Findings from the simulations reveal that when the proportion of items with the slipping effects is small (20%), ignoring misfit does not have practical importance; however, when the proportion of items with the slipping effects is moderate to large (50%–80%), the abilities are generally underestimated at both ends of ability scale. When an upper asymptote parameter was used for modeling the slipping effects, the items became easier and more discriminative in general than the model ignoring the slipping effects. [ABSTRACT FROM AUTHOR]

In practice, some test items may display misfit at the upper-asymptote of item characteristic curve due to distraction, anxiety, or carelessness by the test takers (i.e., the slipping effect). The conventional item response theory (IRT) models do not take the slipping effect into consideration, which may violate the model fit assumption in IRT. The 4-parameter logistic model (4PLM) includes a parameter for the misfit at the upper-asymptote. Although the 4PLM took more attention by researchers in recent years, there are a few studies on the sample size requirements for the 4PLM in the literature. The current study investigated the sample size requirements for the parameter recovery in the 4PLM with a systematic simulation study design. Results indicated that the item parameters in the 4PLM can be estimated accurately when the sample size is at least 4000, and the person parameters, excluding the extreme ends of the ability scale, can be estimated accurately for the conditions with a sample size of at least 750. [ABSTRACT FROM AUTHOR]

Pseudo-guessing parameters are present in item response theory applications for many educational assessments. When sample size is not sufficiently large, the guessing parameters may be ignored from the analysis. This study examines the impact of ignoring pseudo-guessing parameters on measurement invariance analysis, specifically, on item difficulty, item discrimination, and mean and variance of ability distribution. Results show that when non-zero guessing parameters are ignored from the measurement invariance analysis, item discrimination estimates tend to decrease particularly for more difficult items, and item difficulty estimates decrease unless the items are highly discriminating and difficult. As the guessing parameter increases, the size of the decrease in item discrimination and difficulty tends to increase, and the estimated mean and variance of ability distribution tend to be inaccurate. When two groups have heterogeneous ability distributions, ignoring the guessing parameter affects the reference group and the focal group differently. Implications of result findings are discussed. [ABSTRACT FROM AUTHOR]