Search

Test equating is a statistical procedure to ensure that scores from different test forms are comparable and can be used interchangeably. There are several methodologies available to perform equating, some of which are based on the Classical Test Theory (CTT) framework and others are based on the Item Response Theory (IRT) framework. After a short overview of latent trait models and test equating on Chapter 1, Chapter 2 of this thesis proposes a procedure to compare equating transformations originated from different frameworks. As example, we have compared Item Response Theory Observed-Score Equating (IRTOSE), Kernel Equating (KE) and IRT Kernel Equating (IRTKE) under different scenarios. Our results suggest that IRT methods tend to provide better results than KE even when the data are not generated from IRT processes. KE can provide satisfactory results if a proper pre-smoothing solution can be found, while also being much faster than IRT methods. For daily applications, we recommend observing the sensibility of the results to the equating method, minding the importance of good model fit and meeting the assumptions of the framework. Within the IRT framework, if the statistical modeling of the scores from each test form is performed independently, their respective parameters will be on different scales and thus incomparable. Equating solves this problem by transforming item parameters so they are all on the same scale. Popular IRT methods for equating pairs of test forms include the mean-sigma, mean-mean, Stocking–Lord and Haebara. For multiple forms, it might be necessary to employ more elaborate methods which take into account all the relationships between the forms. Chapter 3 addresses this issue, as we propose a new statistical methodology that simultaneously equates a large number of test forms. Our proposal differentiates itself from the current state of the art by using the likelihood function of the true item parameters and the equating coefficients to perform the simultaneous estimation of all equating coefficients and by taking into account the heteroskedasticity of the item parameter estimates as well as the correlations between those estimates on each test form. Such innovations give this new method the potential to yield equating coefficient estimates which are more efficient than what is currently available in the literature, albeit at a computational cost due to its increased complexity. This is indeed what has been observed in some of the simulations performed. Greater estimation efficiency is especially important in situations involving item parameters with extreme values.

Despite a long tradition in the study of graphs and relational data, for decades the analysis of complex networks was limited by difficulties in data collection and computational burdens. The advent of new technologies in life sciences, as well as in our daily life, has suddenly shed light on the many interconnections that our world features, from friendships and collaborations between individuals or organizations, to functional couplings between cellular molecules. This has highly facilitated the collection of relational data, fostering an unprecedented interest in network science. Understanding relations encoded in complex networks, however, still represents a challenging task, and statistical methods that can help to summarize and simplify complex networks are needed. In this thesis we show that often one can gain a deep insight of a network by focusing their attention on communities, i.e. on clusters of nodes, and on the relations that exist between them. We begin by presenting NEAT, a network-based test that allows to assess relations between gene sets in a gene interaction network. NEAT extends traditional gene enrichment analysis tests by incorporating information on interactions between genes and it overcomes some limitations of existing network enrichment analysis approaches. Then, we propose two extended stochastic blockmodels that allow to infer the relations that exist between communities from relations between pairs of individuals in a social network. We advocate the use of penalized inference to estimate these models, with the aim of deriving a sparse reduced graph between communities. Application of these models to bill cosponsorship networks in the Italian Chamber of Deputies allows us to reconstruct the pattern of collaborations between Italian political parties from 2001 to 2015. Finally, we propose a novel clustering strategy for sequences of graphs, based on mixtures of generalized linear models. We show that the proposed clustering method not only is capable to retrieve subpopulations of networks within a cross-sectional or longitudinal sequence of networks, but it also allows to directly characterize them by considering each of the components that form the mixture model.

Given a big set of data with several variables, the aim is the evaluation of each unit with a method that produces a synthetic measure describing a complex and non-observable concept; this goal is achieved respecting the characteristics of the variables, specially the measurement scale. The information gathered with partially ordered sets (poset) reflects this aim, because posets depends only on the order relations among the observations, and allows to handle ordinal and dichotomous variables fairly. In this setting, the vector of variables observed on a unit is handled as a unique entity called profile and not as a group of different variables that need to be aggregated. Starting from recent developments in poset theory, this thesis is organized in three parts. The first proposes to obtain a unique indicator combining the values given by the severity measures for evaluation, derived from the fuzzy identification method. The second contribution is the HOGS (Height Of Groups by Sampling) procedure, which is aimed to estimate the average rank of groups of units of a big population. HOGS is a step forward the statistical estimation of the average rank of a profile; furthermore it allows the estimation of the effect of external variables on the synthetic measure. The last results are two new R functions: the first computes the approximated average rank for large data sets overcoming the usual sample sizes considered by the software usually used until now, the second implements the information given by the frequency of profiles in the computation of approximated average rank, making its use more profitable for social sciences.

Item response theory parameters have to be estimated, and because of the estimation process, they do have uncertainty in them. In most large-scale testing programs, the parameters are stored in item banks, and automated test assembly algorithms are applied to assemble operational test forms. These algorithms treat item parameters as fixed values, and uncertainty is not taken into account. As a consequence, resulting tests might be off target or less informative than expected. In this article, the process of parameter estimation is described to provide insight into the causes of uncertainty in the item parameters. The consequences of uncertainty are studied. Besides, an alternative automated test assembly algorithm is presented that is robust against uncertainties in the data. Several numerical examples demonstrate the performance of the robust test assembly algorithm, and illustrate the consequences of not taking this uncertainty into account. Finally, some recommendations about the use of robust test assembly and some directions for further research are given.

In testing, item response theory models are widely used in order to estimate item parameters and individual abilities. However, even unidimensional models require a considerable sample size so that all parameters can be estimated precisely. The introduction of empirical prior information about candidates and items might reduce the number of candidates needed for parameter estimation. Using data for IQ measurement, this work shows how empirical information about items can be used effectively for item calibration and in adaptive testing. First, we propose multivariate regression trees to predict the item parameters based on a set of covariates related to the item-solving process. Afterwards, we compare the item parameter estimation when tree-fitted values are included in the estimation or when they are ignored. Model estimation is fully Bayesian, and is conducted via Markov chain Monte Carlo methods. The results are two-fold: (a) in item calibration, it is shown that the introduction of prior information is effective with short test lengths and small sample sizes and (b) in adaptive testing, it is demonstrated that the use of the tree-fitted values instead of the estimated parameters leads to a moderate increase in the test length, but provides a considerable saving of resources.

In this paper, empirical prior information about the candidate is applied in computerized adaptive testing (CAT). The main objective of CAT is to improve efficiency of test administration. In this paper, it is shown how the inclusion of background variables both in the initialization and the ability estimation is able to improve the accuracy of ability estimates. In particular, a Gibbs sampler scheme is proposed in the phases of interim and final ability estimation. By using both simulated and real data, it is demonstrated that the method produces more accurate ability estimates, especially for short tests and when reproducing boundary abilities. This implies that operational problems of CAT related to weak measurement precision under particular conditions, can be reduced as well. In the empirical example, the methods were applied to CAT for intelligence testing in the area of personnel selection. Other promising applications would be in the medical world, where testing efficiency is of paramount importance as well.

In this paper, we investigate the pattern of longevity during the last 15 years in Emilia Romagna, a North-Eastern region of Italy, at a municipality level. We consider a specific index of extreme longevity based on people aged 95 and over in two different periods (1995-1999 and 2005-2009). Spatio-temporal modeling is used to tackle at both periods the random variations in the occurrence of people 95+, due to the increasingly rareness of such events, especially in small areas. This method exploits the spatial proximity and the consequent interaction of the geographical areas to smooth the observations, as well as to control for the effects of a set of regressors. As a result, clusters of areas characterized by high and low indexes of longevity are well identified and the temporal evolution of the phenomenon can be depicted. In a parallel analysis, we consider the past levels of mortality on the same cohort of individuals reaching 95 years and over in the second period and when they were aged 80-89 and 90-99. Within this longitudinal framework, the longevity outcome is modeled by a spatial regression. The area-specific structures of mortality are included as regressors, whose effects represent the causal link between the occurrence of people 95+ and the causes of death in the same cohort.

Il lavoro si propone di ripercorrere alcune metodologie generali di analisi dei test per la valutazione degli apprendimenti, discutendo i risultati ottenuti in due casi di studio riguardanti le prove preparate dal Servizio Nazionale di Valutazione (SNV) dell’INVALSI per la classe seconda della scuola primaria. In particolare, viene descritto il processo di analisi dei pre-test attraverso l’utilizzo congiunto degli indicatori derivanti dalla Classical Test Theory e dei modelli di Item Response Theory.

The paper deals with the investigation of gender differences in performances in mathematics for Italian students at the end of lower secondary school in the Emilia-Romagna region. The study is based on a new large-scale assessment test developed and administered by the National Evaluation Institute for the School System (INVALSI). Given the evidence in the literature which favours males, performances of female and male students are compared using different approaches. In particular, scores proposed by educational experts based on item subgroups were considered, while a model-based approach was used within item response theory to estimate ability scores. The results revealed a significant advantage to males compared with females in overall performance, while no meaningful differences were observed with respect to item domain and item type. Following this, an interpretable item map was developed crossing expert opinions with IRT abilities, and plausible proficiency levels were defined. According to the map-based student classification, a relatively lower percentage of females fell into the highest proficiency groups with respect to males.

Il rapporto presenta i risultati dell'analisi statistiche effettuate sulle risposte date dagli studenti al'Esame di Stato della scuola secondaria di primo grado, classe terza, anno 2007-2008. I dati forniti dall'Invalsi riguardano le risposte fornite alle prove standardizzate di italiano e matematica.

In this paper, empirical prior information is introduced in computer adaptive testing. Despite its increasing use, the method suffers from a weak measurement precision, especially under particular conditions. Therefore, it is shown how the inclusion of background variables both in the initialization and the ability estimation is able to improve the accuracy of ability estimates. In particular, a Gibbs sampler scheme is proposed in the phases of interim and final ability estimation. By using simulated data, it is demonstrated that the method produces more accurate ability estimates, especially for short tests and when reproducing boundary abilities.

When the goal of a study is to assess and compare the evolution of old mortality, the researchers often face with problems of defects in data, such as loss of accuracy and numerical inconsistency of information. In the present work, an adaptive procedure for the issue of survival estimations is proposed as a combination of several methods: spline interpolation to split grouped data into age-specific quantities; use of mortality models for estimating the risk of death at advanced ages; methods for choosing the fitting age range based on the analysis of life-table aging rates. The procedure is applied to Emilia-Romagna data in order to construct complete period life tables corresponding to the occurrence of each population census from 1871 to 2001, and, finally, to compare the measures of old-age mortality by time and sex.