Search

Objective: The aims were to review practices concerning Differential Item Functioning (DIF) detection in composite measurement scales, particularly those used in health research, and to provide guidance on how to proceed if statistically significant DIF is detected., Methods: This work specifically addressed the Rasch model which is the subject of growing interest in the field of health owing to its particularly advantageous properties. There were three steps: 1) Literature review to describe current practices; 2) Simulation study to determine under which conditions encountered in health research studies can erroneous conclusions be drawn from group comparisons when a scale is affected by DIF but which is not considered; 3) Based on steps 1 and 2, formulation of recommendations that were subsequently reviewed by leading internationally recognized experts., Results: Four key recommendations were formulated to help researchers to determine whether statistically significant DIF is meaningful in practice, according to the kind of DIF (uniform or non-uniform) and the DIF effect size., Conclusion: This work provides the first recommendations on how to deal in practice with the presence of DIF in composite measurement scales used in health research studies., Competing Interests: The authors have declared that no competing interests exist.

Introduction: Since the legalisation of online gambling in France in 2010, gambling operators must implement responsible gambling measures to prevent excessive gambling practices. However, actually there is no screening procedure for identifying problematic gamblers. Although several studies have already been performed using several data sets from online gambling operators, the authors deplored several methodological and clinical limits that prevent scientifically validating the existence of problematic gambling behaviour. The aim of this study is to develop a model for screening excessive gambling practices based on the gambling behaviours observed on French gambling websites, coupled with a clinical validation., Methods and Analysis: The research is divided into three successive stages. All analyses will be performed for each major type of authorised online gambling in France. The first stage aims at defining a typology of users of French authorised gambling websites based on their gambling behaviour. This analysis will be based on data from the Authority for Regulating Online Gambling (ARJEL) and the Française Des Jeux (FDJ). For the second stage aiming at determining a score to predict whether a gambler is problematic or not, we will cross answers from the Canadian Problem Gambling Index with real gambling data. The objective of the third stage is to clinically validate the score previously developed. Results from the screening model will be compared (using sensitivity, specificity, area under the curve, and positive and negative predictive values) with the diagnosis obtained with a telephone clinical interview, including diagnostic criteria for gambling addiction., Ethics and Dissemination: This study was approved by the local Research Ethics Committee (GNEDS) on 25 March 2015. Results will be presented in national and international conferences, submitted to peer-reviewed journals and will be part of a PhD thesis. A final report with the study results will be presented to the ARJEL, especially the final screening model., Trial Registration Number: NCT02415296., Competing Interests: Competing interests: MG-B, JC and GC-B declare that the University Hospital of Nantes has received gambling industry (FDJ and PMU) funding in the form of a sponsorship which supports the gambling section of the BALANCED Unit (the Reference Centre for Excessive Gambling). Scientific independence towards gambling industry operators is warranted. There were no publishing constraints. BP, J-MC and J-BH declare that they have no conflict of interest, (© Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.)

Objective: To determine the conditions in which the estimation of a difference between groups for a construct evaluated using a composite measurement scale is biased if the presence of Differential Item Functioning (DIF) is not taken into account., Methods: Datasets were generated using the Partial Credit Model to simulate 642 realistic scenarios. The effect of seven factors on the bias on the estimated difference between groups was evaluated using ANOVA: sample size, true difference between groups, number of items in the scale, proportion of items showing DIF, DIF-size for these items, position of these items location parameters along the latent trait, and uniform/non-uniform DIF., Results: For uniform DIF, only the DIF-size and the proportion of items showing DIF (and their interaction term) had meaningful effects. The effect of non-uniform DIF was negligible., Conclusion: The measurement bias resulting from DIF was quantified in various realistic conditions of composite measurement scale use.

Objective: This simulation study was designed to provide data on the performance of Oort's procedure (OP) for response shift (RS) detection (regarding type I error, power, and overall performance), according to sample characteristics, at item level. A specific objective was to assess the impact of using different information criteria (IC), as alternatives to the LRT (likelihood-ratio test), for global assessment of RS occurrence., Methods: Responses to five binary items at two times of measurement were simulated. Thirty-six combinations of sample characteristics [sample size (n), "true change," correlations between the two latent variables and presence/absence of uniform recalibration RS (ur)] were considered. A thousand datasets were generated for each combination. RS detection was performed on each dataset following OP. Type I error and power of the global assessment of RS occurrence, as well as overall performance of the OP, were assessed., Results: The estimated type I error was close to 5 % for the LRT and lower than 5 % for the IC. The estimated power was higher for the LRT as compared to the AIC, which was the highest among the other IC. For the LRT, the estimated power for n = 100 and for the combination of n = 200 and ur = 1 item was below 80 %. Otherwise, for other combinations of sample characteristics, the estimated power was above 90 %., Conclusion: For the LRT, higher values of power were estimated compared to IC with appropriate values of type I error. These results were consistent with Oort's proposal to use the LRT as the criterion to assess global RS occurrence.

The analysis of patient-reported outcomes or other psychological traits can be realized using the Rasch measurement model. When the objective of a study is to compare groups of individuals, it is important, before the study, to define a sample size such that the group comparison test will attain a given power. The Raschpower procedure (RP) allows doing so with dichotomous items. The RP is extended to polytomous items. Several computational issues were identified, and adaptations have been proposed. The performance of this new version of RP is assessed using simulations. This adaptation of RP allows obtaining a good estimate of the expected power of a test to compare groups of patients in a large number of practical situations. A Stata module, as well as its implementation online, is proposed to perform the RP. Two versions of the RP for polytomous items are proposed (deterministic and stochastic versions). These two versions produce similar results in all of the tested cases. We recommend the use of the deterministic version, when the measure is obtained using small questionnaires or items with a few number of response categories, and the stochastic version elsewhere, so as to optimize computing time., (Copyright © 2015 John Wiley & Sons, Ltd.)

Background: Patient-reported outcomes (PRO) are important as endpoints in clinical trials and epidemiological studies. Guidelines for the development of PRO instruments and analysis of PRO data have emphasized the need to report methods used for sample size planning. The Raschpower procedure has been proposed for sample size and power determination for the comparison of PROs in cross-sectional studies comparing two groups of patients when an item reponse model, the Rasch model, is intended to be used for analysis. The power determination of the test of the group effect using Raschpower requires several parameters to be fixed at the planning stage including the item parameters and the variance of the latent variable. Wrong choices regarding these parameters can impact the expected power and the planned sample size to a greater or lesser extent depending on the magnitude of the erroneous assumptions., Methods: The impact of a misspecification of the variance of the latent variable or of the item parameters on the determination of the power using the Raschpower procedure was investigated through the comparison of the estimations of the power in different situations., Results: The power of the test of the group effect estimated with Raschpower remains stable or shows a very little decrease whatever the values of the item parameters. For most of the cases, the estimated power decreases when the variance of the latent trait increases. As a consequence, an underestimation of this variance will lead to an overestimation of the power of the group effect., Conclusion: A misspecification of the item difficulties regarding their overall pattern or their dispersion seems to have no or very little impact on the power of the test of the group effect. In contrast, a misspecification of the variance of the latent variable can have a strong impact as an underestimation of the variance will lead in some cases to an overestimation of the power at the design stage and may result in an underpowered study.

Purpose: Some IRT models have the advantage of being robust to missing data and thus can be used with complete data as well as different patterns of missing data (informative or not). The purpose of this paper was to develop an algorithm for response shift (RS) detection using IRT models allowing for non-uniform and uniform recalibration, reprioritization RS recognition and true change estimation with these forms of RS taken into consideration if appropriate., Methods: The algorithm is described, and its implementation is shown and compared to Oort's structural equation modeling (SEM) procedure using data from a clinical study assessing health-related quality of life in 669 hospitalized patients with chronic conditions., Results: The results were quite different for the two methods. Both showed that some items of the SF-36 General Health subscale were affected by response shift, but those items usually differed between IRT and SEM. The IRT algorithm found evidence of small recalibration and reprioritization effects, whereas SEM mostly found evidence of small recalibration effects., Conclusion: An algorithm has been developed for response shift analyses using IRT models and allows the investigation of non-uniform and uniform recalibration as well as reprioritization. Differences in RS detection between IRT and SEM may be due to differences between the two methods in handling missing data. However, one cannot conclude on the differences between IRT and SEM based on a single application on a dataset since the underlying truth is unknown. A next step would be to implement a simulation study to investigate those differences.

Measurement specialists routinely assume examinee responses to test are independent of one another. However, previous research has shown that many tests contain item dependencies, and not accounting for these dependencies leads to misleading estimates of item and person parameters. In this paper, the marginal maximum likelihood estimation in Rasch model with the violation of the local independence is studied. The power of the Wald test on a group effect parameter on the latent traits in cross-sectional studies is examined under the local independence and the local item dependence assumptions. The different results are illustrated with simulation studies.

Patient-reported outcomes (PRO) have gained importance in clinical and epidemiological research and aim at assessing quality of life, anxiety or fatigue for instance. Item Response Theory (IRT) models are increasingly used to validate and analyse PRO. Such models relate observed variables to a latent variable (unobservable variable) which is commonly assumed to be normally distributed. A priori sample size determination is important to obtain adequately powered studies to determine clinically important changes in PRO. In previous developments, the Raschpower method has been proposed for the determination of the power of the test of group effect for the comparison of PRO in cross-sectional studies with an IRT model, the Rasch model. The objective of this work was to evaluate the robustness of this method (which assumes a normal distribution for the latent variable) to violations of distributional assumption. The statistical power of the test of group effect was estimated by the empirical rejection rate in data sets simulated using a non-normally distributed latent variable. It was compared to the power obtained with the Raschpower method. In both cases, the data were analyzed using a latent regression Rasch model including a binary covariate for group effect. For all situations, both methods gave comparable results whatever the deviations from the model assumptions. Given the results, the Raschpower method seems to be robust to the non-normality of the latent trait for determining the power of the test of group effect.

Statistics literature in the social, behavioral, and biomedical sciences typically stress the importance of power analysis. Patient Reported Outcomes (PRO) such as quality of life and other perceived health measures (pain, fatigue, stress,...) are increasingly used as important health outcomes in clinical trials or in epidemiological studies. They cannot be directly observed nor measured as other clinical or biological data and they are often collected through questionnaires with binary or polytomous items. The Rasch model is the well known model in the item response theory (IRT) for binary data. The article proposes an approach to evaluate the statistical power of the time effect for the longitudinal Rasch model with two time points. The performance of this method is compared to the one obtained by simulation study. Finally, the proposed approach is illustrated on one subscale of the SF-36 questionnaire.

Evaluation of patient-reported outcomes (PRO) is increasingly performed in health sciences. PRO differs from other measurements because such patient characteristics cannot be directly observed. Item response theory (IRT) is an attractive way for PRO analysis. However, in the framework of IRT, sample size justification is rarely provided or ignores the fact that PRO measures are latent variables with the use of formulas developed for observed variables. It might therefore be inappropriate and might provide inadequately sized studies. The objective was to develop valid sample size methodology for the comparison of PRO in two groups of patients using IRT. The proposed approach takes into account questionnaire's items parameters, the difference of the latent variables means, and its variance whose derivation is approximated using Cramer-Rao bound (CRB). We also computed the associated power. We realized a simulation study taking into account sample size, number of items, and value of the group effect. We compared power obtained from CRB with the one obtained from simulations (SIM) and with the power based on observed variables (OBS). For a given sample size, powers using CRB and SIM were similar and always lower than OBS. We observed a strong impact of the number of items for CRB and SIM, the power increasing with the questionnaire's length but not for OBS. In the context of latent variables, it seems important to use an adapted sample size formula because the formula developed for observed variables seems to be inadequate and leads to an underestimated study size., (Copyright © 2011 John Wiley & Sons, Ltd.)

Background: Nowadays, more and more clinical scales consisting in responses given by the patients to some items (Patient Reported Outcomes - PRO), are validated with models based on Item Response Theory, and more specifically, with a Rasch model. In the validation sample, presence of missing data is frequent. The aim of this paper is to compare sixteen methods for handling the missing data (mainly based on simple imputation) in the context of psychometric validation of PRO by a Rasch model. The main indexes used for validation by a Rasch model are compared., Methods: A simulation study was performed allowing to consider several cases, notably the possibility for the missing values to be informative or not and the rate of missing data., Results: Several imputations methods produce bias on psychometrical indexes (generally, the imputation methods artificially improve the psychometric qualities of the scale). In particular, this is the case with the method based on the Personal Mean Score (PMS) which is the most commonly used imputation method in practice., Conclusions: Several imputation methods should be avoided, in particular PMS imputation. From a general point of view, it is important to use an imputation method that considers both the ability of the patient (measured for example by his/her score), and the difficulty of the item (measured for example by its rate of favourable responses). Another recommendation is to always consider the addition of a random process in the imputation method, because such a process allows reducing the bias. Last, the analysis realized without imputation of the missing data (available case analyses) is an interesting alternative to the simple imputation in this context.

Health sciences frequently deal with Patient Reported Outcomes (PRO) data for the evaluation of concepts, in particular health-related quality of life, which cannot be directly measured and are often called latent variables. Two approaches are commonly used for the analysis of such data: Classical Test Theory (CTT) and Item Response Theory (IRT). Longitudinal data are often collected to analyze the evolution of an outcome over time. The most adequate strategy to analyze longitudinal latent variables, which can be either based on CTT or IRT models, remains to be identified. This strategy must take into account the latent characteristic of what PROs are intended to measure as well as the specificity of longitudinal designs. A simple and widely used IRT model is the Rasch model. The purpose of our study was to compare CTT and Rasch-based approaches to analyze longitudinal PRO data regarding type I error, power, and time effect estimation bias. Four methods were compared: the Score and Mixed models (SM) method based on the CTT approach, the Rasch and Mixed models (RM), the Plausible Values (PV), and the Longitudinal Rasch model (LRM) methods all based on the Rasch model. All methods have shown comparable results in terms of type I error, all close to 5 per cent. LRM and SM methods presented comparable power and unbiased time effect estimations, whereas RM and PV methods showed low power and biased time effect estimations. This suggests that RM and PV methods should be avoided to analyze longitudinal latent variables., (Copyright © 2010 John Wiley & Sons, Ltd.)

Background: Patients-Reported Outcomes (PRO) are increasingly used in clinical and epidemiological research. Two main types of analytical strategies can be found for these data: classical test theory (CTT) based on the observed scores and models coming from Item Response Theory (IRT). However, whether IRT or CTT would be the most appropriate method to analyse PRO data remains unknown. The statistical properties of CTT and IRT, regarding power and corresponding effect sizes, were compared., Methods: Two-group cross-sectional studies were simulated for the comparison of PRO data using IRT or CTT-based analysis. For IRT, different scenarios were investigated according to whether items or person parameters were assumed to be known, to a certain extent for item parameters, from good to poor precision, or unknown and therefore had to be estimated. The powers obtained with IRT or CTT were compared and parameters having the strongest impact on them were identified., Results: When person parameters were assumed to be unknown and items parameters to be either known or not, the power achieved using IRT or CTT were similar and always lower than the expected power using the well-known sample size formula for normally distributed endpoints. The number of items had a substantial impact on power for both methods., Conclusion: Without any missing data, IRT and CTT seem to provide comparable power. The classical sample size formula for CTT seems to be adequate under some conditions but is not appropriate for IRT. In IRT, it seems important to take account of the number of items to obtain an accurate formula.

Sequential methods allowing for early stopping of clinical trials are widely used in various therapeutic areas. These methods allow for the analysis of different types of endpoints (quantitative, qualitative, time to event) and often provide, in average, substantial reductions in sample size as compared with single-stage designs while maintaining pre-specified type I and II errors. Sequential methods are also used when analysing particular endpoints that cannot be directly measured, such as depression, quality of life, or cognitive functioning, which are often measured through questionnaires. These types of endpoints are usually referred to as latent variables and should be analysed with latent variable models. In addition, in most clinical trials studying such latent variables, incomplete data are not uncommon and the missing data process might also be non-ignorable. We investigated the impact of informative or non-informative missing data on the statistical properties of the double triangular test (DTT), combined with the mixed-effects Rasch model (MRM) for dichotomous responses or the traditional method based on observed patient's scores (S) to the questionnaire. The achieved type I errors for the DTT were usually close to the target value of 0.05 for both methods, but increased slightly for the MRM when informative missing data were present. The DTT was very close to the nominal power of 0.95 when the MRM was used, but substantially underpowered with the S method (reduction of about 23 per cent), irrespective of whether informative missing data were present or not. Moreover, the DTT using the MRM allowed for reaching a conclusion (under H(0) or H(1)) with fewer patients than the S method, the average sample number for the latter increasing importantly when the proportion of missing data increased. Incorporating MRM in sequential analysis of latent variables might provide a more powerful method than the traditional S method, even in the presence of non-informative or informative missing data.