Search

Il recente sviluppo delle tecnologie informatiche ha consentito agli istituti di valutazione di migliorare il processo di assemblaggio dei test tramite l’automated test assembly (ATA). Una struttura generale per ATA consiste nell’adottare modelli di programmazione intera-mista. Questi modelli sono pensati per essere risolti dasolver commerciali che, nonostante il loro successo nella gestione della maggiorparte dei problemi noti, non sono sempre in grado di risolvere problemi di ATA molto vincolati o di grandi dimensioni. Inoltre, tutti i parametri sono considerati fissi e noti, un’ipotesi che non vale per le stime dei parametri di item response theory (IRT). In questo lavoro proponiamo un modello chance-constrained per affrontarel’incertezza nei modelli di ATA senza aumentarne la complessità The recent development of computer technologies enabled test institutes to improve the test assembly process by automated test assembly (ATA). A general framework for ATA consists in adopting mixed-integer programming models. These models are intended to be solved by common commercial solvers which, notwithstanding their success in handling most of the known problems, are not always able to find solutions for highly constrained and large-sized ATA problems. Moreover, all parameters are assumed to be fixed and known, a hypothesis that is not true for estimates of item response theory (IRT) parameters. In this work, we propose a chance-constrained model for dealing with uncertainty in ATA without increasing the complexity of the model.

Un approccio longitudinale per l'analisi delle prove Invalsi di matematica: cosa ci può dire sugli studenti in difficoltà

In this study, a Bayesian multidimensonal item response theory (IRT) model assuming the presence of correlated general and specific latent traits is proposed for the investigation of residents' perceptions toward the tourism industry. Data collected in 2012 in the Italian Romagna area were used to study the perceived benefits and costs related to tourism. By using posterior predictive model checks and Bayesian deviance, the additive IRT model was found to fit the data well. More importantly, the results could be interpreted meaningfully, showing the most and least important perceived advantages and disadvantages of tourism for the local community. Finally, thanks to the compensatory structure of the model, the different influence of the overall attitude and the specific perceptions of the respondents could be investigated for each aspect included in the questionnaire.

The overall aim of this work is to review test equating methods with a particularly detailed description of item response theory (IRT) equating. Test score equating is used to compare different test scores from different test forms. Several methods have been developed to conduct equating: traditional methods, kernel method, and IRT equating. We synthetically explain the traditional equating methods which include mean equating, linear equating and equipercentile equating and which have been developed under all the possible data collection designs. We also briefly describe the idea of the kernel method: this is a unified approach to test equating for which recent interesting developments have been proposed. Then we focus on IRT equating, by describing old and new methods: in particular, we define IRT observed-score kernel equating and IRT observed-score equating using covariates, as well as other recent proposals in this field. We conclude the review by describing strengths and weaknesses of the different discussed approaches and by identifying future research topics., Statistica, Vol 77, No 4 (2017)

This paper presents a longitudinal analysis of the outcomes of the Italian national standardized mathematics tests. By intertwining quantitative and qualitative methods, we selected and analysed a set of linked questions among the tests administered to the same cohort of students first in grade 6 and then in grade 8. In particular, we focus on poor knowledge students and we argue an example of the analysis of two linked questions about graphical representation of fractions. The comparison between the two questions allows us to interpret some difficulties of students and to expect possible future behaviours.

Seventh International Workshop on Simulation, 21-25 May, 2013, Department of Statistical Sciences, Unit of Rimini, University of Bologna, Italy. Book of Abstracts

In the field of educational measurement, it often happens that tests consist of different subscales involving explicitly several dimensions. With the aim of estimating the test item psychometric properties and the candidate ability, item response theory (IRT) models are often used with multidimensional structure. In this work, a comparison among multidimensional models with different ability structures is conducted in a case study. In particular, data from Italian student assessments conducted at the end of the lower secondary school by INVALSI are taken into account.

In this paper, we propose the introduction of power priors in the Bayesian estimation of item response theory models. Within this approach, information coming from historical data can be used for the estimation of model parameters based on current data. In the literature, power priors have been discussed for generalized linear models. In this work, power priors are introduced as informative priors at item parameter and ability sampling steps within a Gibbs sampler scheme. By using data from the Hospital Anxiety and Depression Scale (HADS), the efficiency of this approach is demonstrated in terms of measurement precision with small samples.

The aim of the paper is to outline how different psychometric methods can be used for item analysis in the field of standardized educational assessment, with reference to an Italian case study. The research is stimulated by the increasing relevance acknowledged to standardized tests within the Italian context in the last few years. Further, the authors believe that conducting a rigorous analysis of pretested items represents a fundamental requirement for a fair and effective learning evaluation. In particular, the joint use of indicators derived from classical test theory and item response theory models is discussed. The main results of analysis of pretested items for the 2009 Italian National Test of reading comprehension in the second grade of primary school show that, despite the two methods being consistent, both proved important in highlighting different item properties and in detecting weak items.

Recently, Bayesian estimation of item response theory (IRT) models via Markov chain Monte Carlo methods has become very popular. The main reason is that this method is free from the limitations of using Gaussian quadrature in marginal maximum likelihood estimation and it is more easily extendable for the estimation of models with complex structures. Moreover, a Bayesian approach allows the incorporation of dependencies among variables and sources of uncertainty. The role of prior distributions is very important in Bayesian statistics, and informative prior distributions can be used in order to improve the accuracy of parameter estimation under particular conditions, for example when the sample size is small. This last aspect is too often disregarded by researchers, who are used to include in the model flat, uninformative priors, especially for item parameters. Differently, this work shows how the introduction of informative prior distributions, even empirical, is effective in improving the accuracy of model estimation. In particular, we will consider the introduction of prior information on the item parameters, which are treated as random variables within a Bayesian approach. Including informative prior distributions at item level does not state the matter of fairness, as it is argued in case priors are used at ability level. The use of empirical priors is also discussed, with respect to intelligence test data.

In educational and psychological measurement, the study of test dimensionality is a fundamental issue. The aim of the paper is to compare item response theory (IRT) models with different structures in the latent abilites, in the case of a test consisting of different subscales. In fact, a unidimensional model may not properly fit the data while multidimensional models, also assuming the existence of general and specific traits, may be more accurate. IRT models with a multidimensional ability structure are compared through simulation studies in order to understand their effective capability of recovering different test structures. Model estimation is conducted via Markov chain Monte Carlo (MCMC) methods, adopting a fully Bayesian approach. An application is also conducted on real data on Italian standardized student assessments.

In this paper, empirical prior information about the candidate is applied in computerized adaptive testing (CAT). The main objective of CAT is to improve efficiency of test administration. In this paper, it is shown how the inclusion of background variables both in the initialization and the ability estimation is able to improve the accuracy of ability estimates. In particular, a Gibbs sampler scheme is proposed in the phases of interim and final ability estimation. By using both simulated and real data, it is demonstrated that the method produces more accurate ability estimates, especially for short tests and when reproducing boundary abilities. This implies that operational problems of CAT related to weak measurement precision under particular conditions, can be reduced as well. In the empirical example, the methods were applied to CAT for intelligence testing in the area of personnel selection. Other promising applications would be in the medical world, where testing efficiency is of paramount importance as well.

Il volume presenta un’analisi dedicata alla statistica descrittiva e quindi all’interpretazione dei dati relativi a fenomeni reali e lo studio inferenziale dei risultati campionari, con alcuni cenni di elementi probabilistici. L’ultimo capitolo è dedicato al modello di regressione lineare semplice con un breve cenno alla regressione multipla. Il testo, indirizzato agli studenti di Scienze Sociali o Politiche, è un eserciziario con una parte teorica esplicativa di base per ogni argomento, per facilitarne la comprensione propone schede teoriche riassuntive e successive applicazioni su dataset con il foglio elettronico Excel.

Aspetti statistici per l’analisi e la lettura dei risultati PISA

Il lavoro si propone di ripercorrere alcune metodologie generali di analisi dei test per la valutazione degli apprendimenti, discutendo i risultati ottenuti in due casi di studio riguardanti le prove preparate dal Servizio Nazionale di Valutazione (SNV) dell’INVALSI per la classe seconda della scuola primaria. In particolare, viene descritto il processo di analisi dei pre-test attraverso l’utilizzo congiunto degli indicatori derivanti dalla Classical Test Theory e dei modelli di Item Response Theory.

Il lavoro si propone di ripercorrere alcune metodologie generali di analisi dei test per la valutazione degli apprendimenti, discutendo i risultati ottenuti in due casi di studio riguardanti le prove preparate dal Servizio Nazionale di Valutazione (SNV) dell’INVALSI per la classe seconda della scuola primaria. In particolare, viene descritto il processo di analisi dei pre-test attraverso l’utilizzo congiunto degli indicatori derivanti dalla Classical Test Theory e dei modelli di Item Response Theory.

The paper deals with the investigation of gender differences in performances in mathematics for Italian students at the end of lower secondary school in the Emilia-Romagna region. The study is based on a new large-scale assessment test developed and administered by the National Evaluation Institute for the School System (INVALSI). Given the evidence in the literature which favours males, performances of female and male students are compared using different approaches. In particular, scores proposed by educational experts based on item subgroups were considered, while a model-based approach was used within item response theory to estimate ability scores. The results revealed a significant advantage to males compared with females in overall performance, while no meaningful differences were observed with respect to item domain and item type. Following this, an interpretable item map was developed crossing expert opinions with IRT abilities, and plausible proficiency levels were defined. According to the map-based student classification, a relatively lower percentage of females fell into the highest proficiency groups with respect to males.

Il rapporto presenta i risultati dell'analisi statistiche effettuate sulle risposte date dagli studenti al'Esame di Stato della scuola secondaria di primo grado, classe terza, anno 2007-2008. I dati forniti dall'Invalsi riguardano le risposte fornite alle prove standardizzate di italiano e matematica.