Search

This paper describes how the Rasch measurement model was used to develop an assessment instrument designed to measure student ability to write law enforcement incident and investigative reports. The ability to write reports is a requirement of all law enforcement recruits in the state of Michigan and is a part of the state's mandatory basic training curriculum, which is promulgated by the Michigan Commission on Law Enforcement Standards (MCOLES). Recently, MCOLES conducted research to modernize its training and testing in the area of report writing. A structured validation process was used, which included: a) an examination of the job tasks of a patrol officer, b) input from content experts, c) a review of the professional research, and d) the creation of an instrument to measure student competency. The Rasch model addressed several measurement principles that were central to construct validity, which were particularly useful for assessing student performances. Based on the results of the report writing validation project, the state established a legitimate connectivity between the report writing standard and the essential job functions of a patrol officer in Michigan. The project also produced an authentic instrument for measuring minimum levels of report writing competency, which generated results that are valid for inferences of student ability. Ultimately, the state of Michigan must ensure the safety of its citizens by licensing only those patrol officers who possess a minimum level of core competency. Maintaining the validity and reliability of both the training and testing processes can ensure that the system for producing such candidates functions as intended.

Historically, rule-of-thumb critical values have been employed for interpreting fit statistics that depict anomalous person and item response patterns in applications of the Rasch model. Unfortunately, prior research has shown that these values are not appropriate in many contexts. This article introduces a bootstrap procedure for identifying reasonable critical values for Rasch fit statistics and compares the results of that procedure to applications of rule-of-thumb critical values for three example datasets. The results indicate that rule-of-thumb values may over- or under-identify the number of misfitting items or persons.

One of the most important applications of the Rasch measurement models in educational assessment is the equating of tests. An important feature of attainment tests is the use of both dichotomous and polytomous items. The partial credit model (PCM) developed by Masters (1982) represents an extension of the dichotomous Rasch model for analysing polytomous item data. The dichotomous Rasch model has been used primarily to analyse dichotomous item data. Whilst the partial credit model can provide detailed information on the performance of individual score categories of polytomous items, it is mathematically more complex to use than the dichotomous Rasch model and can, under certain circumstances, present difficulties in interpreting item measures and in practical applications. This study explores the potential of using the dichotomous Rasch model to analyse polytomous items and equate tests. Results obtained from a simulation study and from analysing the data of a science achievement test indicate that the partial credit model and the dichotomous Rasch model produce similar item and person measures and equivalent cut scores on different test forms.

This study aimed to explore the effect on measurement precision of multidimensional, as compared with unidimensional, Rasch measurement for constructing measures from multidimensional Likert-type scales. Many educational and psychological tests are multidimensional but common practice is to ignore correlations among the latent traits in these multidimensional scales in the measurement process. These practices may have serious validity and reliability implications. This study made use of both empirical data from 208,083 students, and simulated data simulated by 24 systematic combinations, each replicated 1000 times, of three conditions, namely, sample size, degree of dimensionality, and scale length to compare unidimensional and multidimensional approaches and to identify effects of sample size, dimensionality and scale length on measurement precision. Results showed that the multidimensional Rasch approach yielded more precise estimates than did unidimensional approach if the two dimensions were strongly correlated. The effect was more pronounced for long scales.

This paper reports on an investigation on the real-life situations students in grades 8 and 9 in South Africa and Albania prefer to use in Mathematics. The functioning of the instrument used to assess the order of preference learners from both countries have for contextual situations is assessed using Rasch modeling techniques. For both the cohorts, the data fit the Rasch model. The differential item functioning (DIF) analysis rendered 3 items operating differentially for the two cohorts. Explanations for these differences are provided in terms of differences in experiences learners in the two countries have related to some of the contextual situations. Implications for interpretation of international comparative tests are offered, as are the possibilities for the cross-country development of curriculum materials related to contexts that learners prefer to use in Mathematics.

This research describes the development process, psychometric analyses and part validation study of a theoretically-grounded Rasch-based instrument, the Nature of Science Instrument-Elementary (NOSI-E). The NOSI-E was designed to measure elementary students' understanding of the Nature of Science (NOS). Evidence is provided for three of the six validity aspects (content, substantive and generalizability) needed to support the construct validity of the NOSI-E. A future article will examine the structural and external validity aspects. Rasch modeling proved especially productive in scale improvement efforts. The instrument, designed for large-scale assessment use, is conceptualized using five construct domains. Data from 741 elementary students were used to pilot the Rasch scale, with continuous improvements made over three successive administrations. The psychometric properties of the NOSI-E instrument are consistent with the basic assumptions of Rasch measurement, namely that the items are well-fitting and invariant. Items from each of the five domains (Empirical, Theory-Laden, Certainty, Inventive, and Socially and Culturally Embedded) are spread along the scale's continuum and appear to overlap well. Most importantly, the scale seems appropriately calibrated and responsive for elementary school-aged children, the target age group. As a result, the NOSI-E should prove beneficial for science education research. As the United States' science education reform efforts move toward students' learning science through engaging in authentic scientific practices (NRC, 2011), it will be important to assess whether this new approach to teaching science is effective. The NOSI-E can be used as one measure of whether this reform effort has an impact.

A questionnaire that evaluates medical students' knowledge of and attitudes towards primary care and family medicine was developed and validated in order to analyze changes during medical training. A 34 items questionnaire with 5 options on a Likert-type scale was designed. Based on this, a 21 item version was developed and validated. The internal consistency, using Cronbach's alpha test, was estimated, and the questionnaire was analyzed by Rasch model. One hundred and fifty nine students responded to the brief questionnaire (95.78% of those enrolled). Cronbach's alpha was 0.72. Rasch analysis showed high adjustment of the items to the pattern, whit an average reliability of the estimates of the items of 0.97 and of the people of 0.68. Our questionnaire has shown acceptable internal consistency and good construct validity.

This study investigated teacher-student relationships from the teachers' point of view at Perth metropolitan schools in Western Australia. The study identified three key social and emotional aspects that affect teacher-student relationships, namely, Connectedness, Availability and Communication. Data were collected by questionnaire (N = 139) with stem-items answered in three perspectives: (1) Idealistic: this is what I would like to happen; (2) Capability: this is what I am capable of; and (3) Behaviour: this is what actually happens, using four ordered response categories: not at all (score 1), some of the time (score 2), most of the time (score 3), and almost always (score 4). Data were analysed with a Rasch measurement model and a uni-dimensional, linear scale with 24 items, ordered from easy to hard, was created. The data were shown to be highly reliable, so that valid inferences could be made from the scale. The Person Separation Index (akin to a reliability index) was 0.93; there was good global teacher and item fit to the measurement model; there was good item fit; the targeting of the item difficulties against the teacher measures was good, and the response categories were answered consistently and logically. Teachers said that the ideal items were all easier than their corresponding capability items which were in turn easier than the behaviour items (where the items fitted the model), as conceptualized. The easiest ideal items were: I like this child and This child and I get along well together. The hardest ideal item (but still easy) was: I am available for this child. The easiest behaviour item (but still hard) was: This child and I get along well together. The hardest behaviour item (and very hard) was: I am interested to learn about this child's personal thoughts, feelings and experiences. The difficulties of the items supported the conceptual structure of the variable.

Understanding language development is incomplete without recognizing children's sociocultural environments, including adult beliefs about language development. Yet there is a need for data supporting valid inferences to assess these beliefs. The current study investigated the psychometric properties of data from a survey (MODeL) designed to explore beliefs in the popular culture, and their alignment with more formal theories. Support for the content, substantive, structural, generalizability, and external aspects of construct validity of the data were investigated. Subscales representing Behaviorist, Cognitive, Nativist, and Sociolinguistic models were identified as dimensions of beliefs. More than half of the items showed a high degree of consensus, suggesting culturally-transmitted beliefs. Behaviorist ideas were most popular. Bilingualism and ethnicity were related to Cognitive and Sociolinguistic beliefs. Identifying these beliefs may clarify the nature of child-directed speech, and enable the design of language intervention programs that are congruent with family and cultural expectations.

Conditional pairwise estimates parameters in Rasch models separate inference on item parameters from inference on person parameters. Pairwise item parameter estimates are consistent when sample size approaches infinity, but adds an extra random error to estimation compared to conditional maximum likelihood estimates. Pairwise estimates of person parameters are easily calculated, but can rarely be assumed to be consistent since the number of items is often small and the properties of the estimates generally unknown. This note gives results from a study of conditional pairwise estimation of person parameters and suggests a modification of the estimate that takes care of some of the error.

The use of performance assessments in which human raters evaluate student achievement has become increasingly prevalent in high-stakes assessment systems such as those associated with recent policy initiatives (e.g., Race to the Top). In this study, indices of rating quality are compared between two measurement perspectives. Within the context of a large-scale writing assessment, this study focuses on the alignment between indices of rater agreement, error, and accuracy based on traditional and Rasch measurement theory perspectives. Major empirical findings suggest that Rasch-based indices of model-data fit for ratings provide information about raters that is comparable to direct measures of accuracy. The use of easily obtained approximations of direct accuracy measures holds significant implications for monitoring rating quality in large-scale rater-mediated performance assessments.

Today, the amount of stress the correctional staff endures at work is an important issue. Research has addressed this issue, but has yielded no consensus as to a properly calibrated measure of perceptions of work stress for correctional staff. Using data from a non-random sample of correctional staff (n = 228), the Rasch model was used to assess whether a specific measure of work stress would fit the model. Results show that three items rather than six items accurately represented correctional staff perceptions of work stress.

We show that extended Rasch models, built to deal with continuous latent variables, may be useful for assessing validity of medical diagnostic tests in the presence of a reference standard, particularly for chronic diseases. We derive estimates for sensitivity and specificity under the Rasch model assumptions. Our estimations can be computed conditionally on the level of potential confounding covariates, making use of a variety of extended Rasch models, namely log-linear Rasch models, to examine the association between covariates and both disease intensity and response to the tests. Also, another variety of extended Rasch models, partial credit models, can determine appropriate thresholds for quantitative diagnostic tests. As an example, we study the validity of some diagnostic tests of heart failure.

Positive attitudes towards teamwork among health care professionals are critical to patient safety. The purpose of this study is to describe the development and concurrent validation of a new instrument to measure attitudes towards healthcare teamwork that is generalizable across various populations of healthcare students. The Collaborative Healthcare Interdisciplinary Planning (CHIRP) scale was validated against the Readiness for Inter-Professional Learning Scale (RIPLS). Analyses included student (n = 266) demographics, ANOVA, internal consistency, factor analysis, and Rasch analysis. The two instruments correlated at r = .582. The CHIRP showed a multifactorial structure having excellent internal consistency (alpha = .850), with 25 of the 36 scale items loading onto a single Teamwork Attitudes factor. The RIPLS likewise had strong internal consistency (alpha = .796) and a three-factor structure, supporting previous studies of the instrument. However, Rasch analyses showed 14 (38.9%) of the 36 CHIRP items, but only four (21.1%) of the 19 RIPLS items remaining within the satisfactory standardized OUTFIT zone of 2.0 standard deviation units. We propose the 14 fitting items as a new, validated teamwork attitudes scale.

jMetrik and WINSTEPS are two Rasch measurement software applications that implement joint maximum likelihood estimation of Rasch, partial credit, and rating scale model parameters via a proportional curve fitting algorithm. We describe this algorithm in this paper and explain the handling of missing data and extreme cases. Results from a simulation study that manipulated sample size and the number of test items indicate that both programs produce similar bias and root mean squared error values. In addition, root mean squared difference values indicate that estimates from each program are within 0.001 and 0.004 logits of each other depending on the model in question.