Search

Four item response theory (IRT) models were compared using data from tests where multiple items were grouped into testlets focused on a common stimulus. In the bi-factor model each item was treated as a function of a primary trait plus a nuisance trait due to the testlet; in the testlet-effects model the slopes in the direction of the testlet traits were constrained within each testlet to be proportional to the slope in the direction of the primary trait; in the polytomous model the item scores were summed into a single score for each testlet; and in the independent-items model the testlet structure was ignored. Using the simulated data, reliability was overestimated somewhat by the independent-items model when the items were not independent within testlets. Under these nonindependent conditions, the independent-items model also yielded greater root mean square error (RMSE) for item difficulty and underestimated the item slopes. When the items within testlets were instead generated to be independent, the bi-factor model yielded somewhat higher RMSE in difficulty and slope. Similar differences between the models were illustrated with real data. [ABSTRACT FROM AUTHOR]

The validity of inferences based on achievement test scores is dependent on the amount of effort that examinees put forth while taking the test. With low-stakes tests, for which this problem is particularly prevalent, there is a consequent need for psychometric models that can take into account differing levels of examinee effort. This article introduces the effort-moderated IRT model, which incorporates item response time into proficiency estimation and item parameter estimation. In two studies of the effort-moderated model when rapid guessing (i.e., reflecting low examinee effort) was present, one based on real data and the other on simulated data, the effort-moderated model performed better than the standard 3PL model. Specifically, it was found that the effort-moderated model (a) showed better model fit, (b) yielded more accurate item parameter estimates, (c) more accurately estimated test information, and (d) yielded proficiency estimates with higher convergent validity. [ABSTRACT FROM AUTHOR]

Data were generated to simulate multidimensionality resulting from including two or four subtopics on a test. Each item was dependent on an ability trait due to instruction and learning, which was the same across all items, as well as an ability trait unique to the subtopic of the test (such as biology on a general science test). The eigenvalues of the item correlation matrix and Yen's Q[sub 3] were not greatly influenced by multidimensionality under conditions where the responses of a large proportion of students shared the influence of common instruction across subtopics. In contrast, Stout's T procedure was effective at detecting this type of multidimensionality, unless the subtopic abilities were correlated. [ABSTRACT FROM AUTHOR]