Search

Previous studies showed that textual information could be used to screen respondents for posttraumatic stress disorder (PTSD). In this study, we explored the feasibility of using language features extracted from short text descriptions respondents provided of stressful events to predict trauma-related symptoms assessed using the Global Psychotrauma Screen. Texts were analyzed with both closed- and open-vocabulary methods to extract language features representing the occurrence of words, phrases, or specific topics in the description of stressful events. We also evaluated whether combining language features with self-report information, including respondents’ demographics, event characteristics, and risk factors for trauma-related disorders, would improve the prediction performance. Data were collected using an online survey on a cross-national sample of 5048 respondents. Results showed that language data achieved the highest predictive power when both closed- and open-vocabulary features were included as predictors. Combining language data and self-report information resulted in a significant increase in performance and in a model which achieved good accuracy as a screener for probable PTSD diagnosis (.7 < AUC ≤ .8), with similar results regardless of the length of the text description of the event. Overall, results indicated that short texts add to the detection of trauma-related symptoms and probable PTSD diagnosis.

Computerized adaptive testing (CAT) comes with many advantages. Unfortunately, it still is quite expensive to develop and maintain an operational CAT. In this paper, various steps involved in developing an operational CAT are described and literature on these topics is reviewed. Bayesian CAT is introduced as an alternative, and the use of empirical priors is proposed for estimating item and person parameters to reduce the costs of CAT. Methods to elicit empirical priors are presented and a two small examples are presented that illustrate the advantages of Bayesian CAT. Implications of the use of empirical priors are discussed, limitations are mentioned and some suggestions for further research are formulated. El test adaptativo Computadorizado (CAT) tiene muchas ventajas. Aunque, infelizmente, es bastante caro desarrollar y mantener un CAT operacional. En este artículo se describen varias etapas de su desarrollo y se hace una revision de literatura del topico. El CAT Bayesiano aparece como una alternativa, y se propone el uso de prioris empiricas para estimar parametros de items y de individuos con el objeto de reducir sus costos. Se presentan metodos para obtener prioris empiricas y dos pequeños ejemplos que ilustran la ventaja del CAT Bayesiano. Se discuten algunas implicaciones en el uso de prioris empiricas, se mencionan limitaciones y se formulan sugerencias para nuevas investigaciónes. O teste adaptativo Computadorizado (CAT) chega com muitas vantagens. Infelizmente, ainda e bastante caro para desenvolver e manter um CAT operacional. Neste artigo, descreve-se varias etapas envolvidas no desenvolvimento de um CAT operacional e faz-se uma revisao da literatura nesse topico. O CAT Bayesiano e introduzido como uma alternativa, e propoe-se o uso de prioris empiricas para estimar parâmetros de itens e de individuos com o objetivo de reduzir os custos de CAT. Apresenta-se metodos para obtencao de prioris empiricas e dois pequenos exemplos para ilustrar a vantagem do CAT Bayesiano. Discute-se algumas implicacoes no uso de prioris empiricas, menciona-se limitacoes e formula-se algumas sugestoes para novas pesquisas.

The focus of this article is on the choice of suitable prior distributions for item parameters within item response theory (IRT) models. In particular, the use of empirical prior distributions for item parameters is proposed. Firstly, regression trees are implemented in order to build informative empirical prior distributions. Secondly, model estimation is conducted within a fully Bayesian approach through the Gibbs sampler, which makes estimation feasible also with increasingly complex models. The main results show that item parameter recovery is improved with the introduction of empirical prior information about item parameters, also when only a small sample is available.

In this paper, we propose the introduction of power priors in the Bayesian estimation of item response theory models. Within this approach, information coming from historical data can be used for the estimation of model parameters based on current data. In the literature, power priors have been discussed for generalized linear models. In this work, power priors are introduced as informative priors at item parameter and ability sampling steps within a Gibbs sampler scheme. By using data from the Hospital Anxiety and Depression Scale (HADS), the efficiency of this approach is demonstrated in terms of measurement precision with small samples.

Item response theory parameters have to be estimated, and because of the estimation process, they do have uncertainty in them. In most large-scale testing programs, the parameters are stored in item banks, and automated test assembly algorithms are applied to assemble operational test forms. These algorithms treat item parameters as fixed values, and uncertainty is not taken into account. As a consequence, resulting tests might be off target or less informative than expected. In this article, the process of parameter estimation is described to provide insight into the causes of uncertainty in the item parameters. The consequences of uncertainty are studied. Besides, an alternative automated test assembly algorithm is presented that is robust against uncertainties in the data. Several numerical examples demonstrate the performance of the robust test assembly algorithm, and illustrate the consequences of not taking this uncertainty into account. Finally, some recommendations about the use of robust test assembly and some directions for further research are given.

Recently, Bayesian estimation of item response theory (IRT) models via Markov chain Monte Carlo methods has become very popular. The main reason is that this method is free from the limitations of using Gaussian quadrature in marginal maximum likelihood estimation and it is more easily extendable for the estimation of models with complex structures. Moreover, a Bayesian approach allows the incorporation of dependencies among variables and sources of uncertainty. The role of prior distributions is very important in Bayesian statistics, and informative prior distributions can be used in order to improve the accuracy of parameter estimation under particular conditions, for example when the sample size is small. This last aspect is too often disregarded by researchers, who are used to include in the model flat, uninformative priors, especially for item parameters. Differently, this work shows how the introduction of informative prior distributions, even empirical, is effective in improving the accuracy of model estimation. In particular, we will consider the introduction of prior information on the item parameters, which are treated as random variables within a Bayesian approach. Including informative prior distributions at item level does not state the matter of fairness, as it is argued in case priors are used at ability level. The use of empirical priors is also discussed, with respect to intelligence test data.

In testing, item response theory models are widely used in order to estimate item parameters and individual abilities. However, even unidimensional models require a considerable sample size so that all parameters can be estimated precisely. The introduction of empirical prior information about candidates and items might reduce the number of candidates needed for parameter estimation. Using data for IQ measurement, this work shows how empirical information about items can be used effectively for item calibration and in adaptive testing. First, we propose multivariate regression trees to predict the item parameters based on a set of covariates related to the item-solving process. Afterwards, we compare the item parameter estimation when tree-fitted values are included in the estimation or when they are ignored. Model estimation is fully Bayesian, and is conducted via Markov chain Monte Carlo methods. The results are two-fold: (a) in item calibration, it is shown that the introduction of prior information is effective with short test lengths and small sample sizes and (b) in adaptive testing, it is demonstrated that the use of the tree-fitted values instead of the estimated parameters leads to a moderate increase in the test length, but provides a considerable saving of resources.

A major goal in computerized learning systems is to optimize learning, while in computerized adaptive tests (CAT) efficient measurement of the proficiency of students is the main focus. There seems to be a common interest to integrate computerized adaptive item selection in learning systems and testing. Item selection is a well founded building block of CAT. However, there are a number of problems that prevent the application of a standard approach, based on item response theory, of computerized adaptive item selection to learning systems. In this work attention will be paid to three unresolved points: item banking, item selection, and choice of IRT model. All problems will be discussed, and an approach to automated item bank generation is presented. Finally some recommendations are given.

In this work, the issue of using prior information in test administration is taken into account. The focus is on the development of procedures to include background variables which are strongly related to the latent ability, adopting a Bayesian approach. Because the desirability of prior information for the ability estimation in item response modelling depends on the goals of the test, only some kinds of educational tests might profit of this approach. The procedures will be evaluated in an empirical context and some recommendations about the use of prior information will be given.

Item response theory (IRT) models have been developed in order to study the individual responses to a set of items designed to measure latent abilities. IRT is a measurement theory that was first formalized in the sixties with the fundamental work of Lord and Novick (1968) and it has a predominant role in educational assessment. An IRT model describes the relationship between the observable examinee’s performance in the test, typically in the form of responses to categorical items, and the unobservable latent ability. Therefore, IRT models can be included in the more general framework of latent variable modelling . IRT is used in all phases of test administration, from the test calibration to the estimation of individual abilities, in which the estimated item parameters are used to characterize the examinees. After a brief presentation of the main assumptions of IRT models, several aspects related to specific problems in the context of test administration are treated. Many advances have been introduced over the last few years, that allow both to support more complex models and to improve the esti-mation algorithms. In particular, issues on multidimensionality incomplete design and the inclusion of prior information are discussed, referring both to current literature and to some contributions of the authors. Particular attention is given to the use of the Gibbs sam-pler, in the Markov Chain Monte Carlo (MCMC) methods, for the estimation of IRT models. Finally, an application related to one of this topics is presented in the context of educational assessment

In this work, the use of empirical prior information in computerized adaptive testing (CAT) is investigated. The problem pertains the Bayesian ability estimation when collateral information about the candidates is available. The inclusion of prior information in CAT administration or, when problems of fairness arise, only in the initialization of the algorithm, is evaluated in order to improve the measurement precision.