Search

The purpose of this study was to model expert judgment in test and instrument development using the many-faceted Rasch model. A 150-item value orientation inventory-2 (VOI-2) assessing the value of physical education curriculum goals was developed and evaluated by 128 university educators and 103 school-based physical educators. The experts were asked to rate the consistency of each item to represent one part of the broad curriculum goals using a 5-point rating scale. The many-faceted Rasch model was used to calibrate the rating scores, and 6 facets -- gender, ethnicity, employment type, rater, content area, and item -- were defined. Severity and consistency of the experts' judgments were examined and corrected before being applied to item evaluation. Further, the impact of group membership on expert judgment was examined. Items were then evaluated based on their logit scores and the consistency of their performance. Results suggest that most VOI-2 items were content domain representative and the raters were truly experts. The many-faceted Rasch model demonstrates a psychometrically appropriate technique for applying expert judgment in test development. [ABSTRACT FROM AUTHOR]

Expectancy-Value Questionnaire (EVQ) measures student expectancy beliefs and task values of the domain content (Eccles & Wigfield, 1995).In this study the authors examine measurement invariance of EVQ in the domain of physical education between elementary and middle-school students. Participants included 811 students (3rd–5th grades) from 13 elementary and 903 (6th–8th grades) from 13 middle schools. Students completed the EVQ in their physical education classes within the same semester. A series of hierarchical steps for testing the measurement invariance were conducted based on means and covariance structures. The results suggest that the questionnaire possesses configural and metric invariance, but noninvariant item intercepts between these two groups of students (∆CFI > .05). EVQ can be used to measure expectancy-value constructs in physical education for both middle-school and elementary students. Yet the noninvariant item intercepts posted questions on how the measured construct difference should be interpreted. [ABSTRACT FROM AUTHOR]