Search

When constructed response test items are scored by more than one rater, the repeated ratings allow for the consideration of individual rater bias and variability in estimating student proficiency. Several hierarchical models based on item response theory have been introduced to model such effects. In this article, the authors demonstrate how these models may be extended to include covariates of the rating process. For example, how do features of an essay grader's training affect his or her performance? The authors show how to include covariates by embedding a linear model at appropriate levels of the model hierarchy. Depending on the level, such covariates may be thought of as determining fixed effects or random effects on the rating process. The authors also discuss the appropriate design matrix for such covariates, discuss how to incorporate needed identifiability constraints, and illustrate the methods using data from a rating study of a student assessment. (Contains 9 tables.)

Unfolding response models, a class of item response theory (IRT) models that assume a unimodal item response function (IRF), are often used for the measurement of attitudes. Verhelst and Verstralen (1993)and Andrich and Luo (1993) independently developed unfolding response models by relating the observed responses to a more common monotone IRT model using a latent response model (LRM; Maris, 1995). This article generalizes their approach, and suggests a data augmentation scheme for the estimation of any unfolding response model. The article introduces two Markov chain Monte Carlo (MCMC) estimation procedures for the Bayesian estimation of unfolding model parameters; one is a direct implementation of MCMC, and the second utilizes the data augmentation method. We use the estimation procedure to analyze three data sets, one simulated, and two from real attitudinal surveys. (Contains 9 figures and 7 tables.)

Discusses the hierarchical rater model (HRM) of R. Patz (1996) and shows how it can be used to scale examinees and items, model aspects of consensus among raters, and model individual rater severity and consistency effects. Also shows how the HRM fits into the generalizability theory framework. Compares the HRM to the conventional item response theory facets model. (SLD)

Introduces this special issue on nonparametric item response theory and discusses its use in psychological and sociological research. Outlines each of the eight articles in the issue. (SLD)

Presents results from two methods of evaluating monotonicity in item response models: regressing individual item scores on the total test score and on the "rest" score obtained by omitting the selected items from the total test score. Discusses implications for exploratory analysis of dichotomous item response data and the application of the results to polytomous item response data. (SLD)

Extends the basic Markov chain Monte Carlo (MCMC) strategy of R. Patz and B. Junker (1999) for Bayesian inference in complex Item Response Theory settings to address issues such as nonresponse, designed missingness, multiple raters, guessing behaviors, and partial credit (polytomous) test items. Applies the MCMC method to data from the National Assessment of Educational Progress. (SLD)

Demonstrates Markov chain Monte Carlo (MCMC) techniques that are well-suited to complex models with Item Response Theory (IRT) assumptions. Develops an MCMC methodology that can be routinely implemented to fit normal IRT models, and compares the approach to approaches based on Gibbs sampling. Contains 64 references. (SLD)

Stochastic ordering properties are investigated for a broad class of item response theory (IRT) models for which the monotone likelihood ratio does not hold. A taxonomy is given for nonparametric and parametric models for polytomous models based on the hierarchical relationship between the models. (SLD)

Research studies in psychology and education often seek to detect changes or growth in an outcome over a duration of time. This research provides a solution to those interested in estimating latent traits from psychological measures that rely on human raters. Rater effects potentially degrade the quality of scores in constructed response and performance assessments. We develop an extension of the hierarchical rater model (HRM), which yields estimates of latent traits that have been corrected for individual rater bias and variability, for ratings that come from longitudinal designs. The parameterization, called the longitudinal HRM (L-HRM), includes an autoregressive time series process to permit serial dependence between latent traits at adjacent timepoints, as well as a parameter for overall growth. We evaluate and demonstrate the feasibility and performance of the L-HRM using simulation studies. Parameter recovery results reveal predictable amounts and patterns of bias and error for most parameters across conditions. An application to ratings from a study of character strength demonstrates the model. We discuss limitations and future research directions to improve the L-HRM. [ABSTRACT FROM AUTHOR]

Presents a study which discussed the usability and interpretability issues for single-strategy cognitive assessment models. Discussion on the extensions of item response theory (IRT) models for cognitive assessment; Assessments of transitive reasoning in children; Description of two IRT-like cognitive assessment models.

Presents information on nonparametric item response theory (NIRT). Basic assumptions of NIRT; Exploratory data analysis and item and test features; Model-data fit and the explanation of data structure.

Presents information on a study which focused on the monotonocity theory that is essential to applications of nonparametric item response models. Background on the study; Discussion on elementary symmetric functions; Methods of evaluating monotonicity; Implications for exploratory analysis of dichotomous and polytomous item response data.