Search

Purpose: ReQoL-10 and ReQoL-20 have been developed for use as outcome measures with individuals aged 16 and over, experiencing mental health difficulties. This paper reports modelling results from the item response theory (IRT) analyses that were used for item reduction., Methods: From several stages of preparatory work including focus groups and a previous psychometric survey, a pool of items was developed. After confirming that the ReQoL item pool was sufficiently unidimensional for scoring, IRT model parameters were estimated using Samejima's Graded Response Model (GRM). All 39 mental health items were evaluated with respect to item fit and differential item function regarding age, gender, ethnicity, and diagnosis. Scales were evaluated regarding overall measurement precision and known-groups validity (by care setting type and self-rating of overall mental health)., Results: The study recruited 4266 participants with a wide range of mental health diagnoses from multiple settings. The IRT parameters demonstrated excellent coverage of the latent construct with the centres of item information functions ranging from - 0.98 to 0.21 and with discrimination slope parameters from 1.4 to 3.6. We identified only two poorly fitting items and no evidence of differential item functioning of concern. Scales showed excellent measurement precision and known-groups validity., Conclusion: The results from the IRT analyses confirm the robust structure properties and internal construct validity of the ReQoL instruments. The strong psychometric evidence generated guided item selection for the final versions of the ReQoL measures.

Purpose: This paper presents two studies exploring the latent structure of item sets used in the development of the Recovering Quality of Life mental health outcome measures: ReQoL-10 and ReQoL-20., Method: In study 1, 2262 participants completed an initial set of 61 items. In study 2, 4266 participants completed a reduced set of 40 items. Study 2 evaluated two formats of the questionnaires: one version where the items were intermingled and one where the positively worded and negatively worded items were presented as two separate blocks. Exploratory and confirmatory factor analyses were conducted on both datasets where models were specified using ordinal treatment of the item responses. Dimensionality based on the conceptual framework and methods effects reflecting the mixture of positively worded and negatively worded items were explored. Factor invariance was tested across the intermingled and block formats., Results: In both studies, a bi-factor model (study 1: RMSEA = 0.061; CFI = 0.954; study 2: RMSEA = 0.066; CFI = 0.971) with one general factor and two local factors (positively worded questions and negatively worded questions) was preferred. The loadings on the general factor were higher than on the two local factors suggesting that the ReQoL scale scores can be understood in terms of a general factor. Insignificant differences were found between the intermingled and block formats., Conclusions: The analyses confirmed that the ReQoL item sets are sufficiently unidimensional to proceed to item response theory analysis. The model was robust across different ordering of positive and negative items.

Objective: The study aims to develop and validate short versions of the work role functioning questionnaire v2.0 (WRFQ) that retain the measurement properties of the full-length 27-item questionnaire., Study Design and Setting: Six cross-sectional Dutch samples (N = 2,433) were used, containing data on gender, self-rated health, job type, and WRFQ scores. Indicators from classical test theory and item response theory methods were used along with evaluation of translatability and conceptual considerations to identify short version candidate items. To ensure content validity, the item selection was made within the five-factor structure established for the WRFQ-leading to a 5-item and a 10-item short version. Bland-Altman analyses of agreement and interclass correlations with the full WRFQ were used to establish the best scoring procedure. Discriminant validity was evaluated for the short versions and compared with the full-length 27-item version., Results: Both short versions showed acceptable agreement with the full-length 27-item version using simple scoring procedures. Both also showed comparable or stronger validity than the full WRFQ in known group comparisons., Conclusion: Both short versions can be used to measure work role functioning in working samples with mixed clinical conditions and job types., (Copyright © 2018 Elsevier Inc. All rights reserved.)

Background: Outcome measures for mental health services need to adopt a service-user recovery focus. Aims To develop and validate a 10- and 20-item self-report recovery-focused quality of life outcome measure named Recovering Quality of Life (ReQoL)., Method: Qualitative methods for item development and initial testing, and quantitative methods for item reduction and scale construction were used. Data from >6500 service users were factor analysed and item response theory models employed to inform item selection. The measures were tested for reliability, validity and responsiveness., Results: ReQoL-10 and ReQoL-20 contain positively and negatively worded items covering seven themes: activity, hope, belonging and relationships, self-perception, well-being, autonomy, and physical health. Both versions achieved acceptable internal consistency, test-retest reliability (>0.85), known-group differences, convergence with related measures, and were responsive over time (standardised response mean (SRM) > 0.4). They performed marginally better than the Short Warwick-Edinburgh Mental Well-being Scale and markedly better than the EQ-5D., Conclusions: Both versions are appropriate for measuring service-user recovery-focused quality of life outcomes. Declaration of interest M.B. and J.Co. were members of the research group that developed the Clinical Outcomes in Routine Evaluation (CORE) outcome measures.

Objective: To evaluate the extent of differential item functioning (DIF) within the thyroid-specific quality of life patient-reported outcome measure, ThyPRO, according to sex, age, education and thyroid diagnosis., Study Design and Setting: A total of 838 patients with benign thyroid diseases completed the ThyPRO questionnaire (84 five-point items, 13 scales). Uniform and nonuniform DIF were investigated using ordinal logistic regression, testing for both statistical significance and magnitude (∆R(2) > 0.02). Scale level was estimated by the sum score, after purification., Results: Twenty instances of DIF in 17 of the 84 items were found. Eight according to diagnosis, where the goiter scale was the one most affected, possibly due to differing perceptions in patients with auto-immune thyroid diseases compared to patients with simple goiter. Eight DIFs according to age were found, of which 5 were in positively worded items, which younger patients were more likely to endorse; one according to gender: women were more likely to report crying, and three according to educational level. The vast majority of DIF had only minor influence on the scale scores (0.1-2.3 points on the 0-100 scales), but two DIF corresponded to a difference of 4.6 and 9.8, respectively., Conclusion: Ordinal logistic regression identified DIF in 17 of 84 items. The potential impact of this on the present scales was low, but items displaying DIF could be avoided when developing abbreviated scales, where the potential impact of DIF (due to fewer items) will be larger.

Purpose: Thresholds for meaningful within-individual change (MWIC) are useful for interpreting patient-reported outcome measures (PROM). Transition ratings (TR) have been recommended as anchors to establish MWIC. Traditional statistical methods for analyzing MWIC such as mean change analysis, receiver operating characteristic (ROC) analysis, and predictive modeling ignore problems of floor/ceiling effects and measurement error in the PROM scores and the TR item. We present a novel approach to MWIC estimation for multi-item scales using longitudinal item response theory (LIRT). Methods: A Graded Response LIRT model for baseline and follow-up PROM data was expanded to include a TR item measuring latent change. The LIRT threshold parameter for the TR established the MWIC threshold on the latent metric, from which the observed PROM score MWIC threshold was estimated. We compared the LIRT approach and traditional methods using an example data set with baseline and three follow-up assessments differing by magnitude of score improvement, variance of score improvement, and baseline-follow-up score correlation. Results: The LIRT model provided good fit to the data. LIRT estimates of observed PROM MWIC varied between 3 and 4 points score improvement. In contrast, results from traditional methods varied from 2 to 10 points—strongly associated with proportion of self-rated improvement. Best agreement between methods was seen when approximately 50% rated their health as improved. Conclusion: Results from traditional analyses of anchor-based MWIC are impacted by study conditions. LIRT constitutes a promising and more robust analytic approach to identifying thresholds for MWIC. [ABSTRACT FROM AUTHOR]

The PROsetta Stone Project, summarized in this issue by Schalet et al. (Psychometrika 86, 2021), is a major step forward in enabling comparability between different patient-reported outcomes measures. Schalet et al. clearly describe the psychometric methods used in the PROsetta Stone project and other projects from the Patient-Reported Outcomes Measurement Information System (PROMIS): linking based on unidimensional item response theory (IRT), equipercentile linking, and calibrated projection based on multidimensional IRT. Analyses in a validation data set and simulation studies provide strong support that the linking methods are robust when basic assumptions are fulfilled. The links already established will be of great value to the field, and the methodology described by Schalet et al. will hopefully inspire the next series of linking studies. Among potential improvements that should be considered by new studies are: (1) a thorough evaluation of the content of the measures to be linked to better guide the evaluation of measurement assumptions, (2) improvements in the design of linking studies such as selection of the optimal sample to provide data in the score ranges where linking precision is most critical and using counterbalanced designs to control for order effects. Finally, it may be useful to consider how the linking algorithms are used in subsequent data analyses. Analytic strategies based on plausible values or latent regression IRT models may be preferable to the simple transformation of scores from one patient at the time. [ABSTRACT FROM AUTHOR]

Background: Patients with rheumatoid arthritis (RA) commonly experience pain despite the availability of disease-modifying treatments. Sleep disturbances are frequently reported in RA, with pain often a contributing factor. The Patient-Reported Outcomes Measurement Information System (PROMIS) Pain Interference and Sleep Disturbance item banks were initially developed to provide insights into the patient experience of pain and sleep, respectively, though they were not specifically intended for use in RA populations. This study evaluated the content validity of the PROMIS Pain Interference and Sleep Disturbance item banks in RA and identified relevant content for short forms for patients with RA that achieved high measurement precision across a broad range of health. Methods: A qualitative approach consisting of hybrid concept elicitation and cognitive debriefing interviews was used to evaluate the content validity of the item banks in RA. Interviews were semi-structured and open-ended, allowing a range of concepts and responses to be captured. Findings from the qualitative interviews were used to select the most relevant items for the short forms, and psychometric evaluation, using existing item-response theory (IRT) item parameters, was used to evaluate the marginal reliability and measurement precision of the short forms across the range of the latent variables (i.e. pain interference and sleep disturbance). Results: Thirty-two participants were interviewed. Participants reported that RA-related pain and sleep disturbances have substantial impacts on their daily lives, particularly with physical functioning. The PROMIS Pain Interference and Sleep Disturbance item banks were easy to understand and mostly relevant to their RA experiences, and the 7-day recall period was deemed appropriate. Qualitative and IRT-based approaches identified short forms for Pain Interference (11 items) and Sleep Disturbance (7 items) that had high relevance and measurement precision, with good coverage of the concepts identified by participants during concept elicitation. Conclusion: Pain and sleep disturbances affect many aspects of daily life in patients with RA and should be considered when novel treatments are developed. This study supports the use of the PROMIS Pain Interference and Sleep Disturbance item banks in RA, and the short forms developed herein have the potential to be used in clinical studies of RA. [ABSTRACT FROM AUTHOR]

Background: Adaptation of a patient-reported outcomes survey into a new language requires careful translation procedures as well as qualitative and quantitative psychometric testing. This study aimed to evaluate the basic psychometric properties of the new Saudi Arabian SF-36v2 and establish norm data for Saudi Arabia. Methods: Translation and adaptation of the SF-36v2 used standard methodology. Psychometric validation included two stages: 1) A qualitative study (n = 100) explored the components of health and health-related quality of life considered important in Saudi Arabia and evaluated the content validity of the SF-36v2 in Saudi Arabia, and 2) A quantitative study (n = 6166) evaluated the basic psychometric properties of the Saudi SF-36v2 and established norm data for Saudi Arabia. Comparison with US general population data (n = 4040) evaluated differential item function (DIF) and cross-national differences. Results: The qualitative study supported the content validity of the Saudi SF-36v2. Cognitive debriefing identified only few and minor problems. Psychometric analyses supported item convergence within scales and differentiation across scales of the SF-36v2. Scale level exploratory factor analyses did not support the typical distinction between physical health and mental health components. Internal consistency reliability was satisfactory for all scales except the social function scale (alpha = 0.67). Cross-national DIF was identified for 9 items. In the Saudi general population, the average vitality score was lower for women (− 2.71 points) compared to men. For men, older age groups scored lower on the physical function scale (− 3.31) and the physical health component (− 3.06). For women, older age groups scored lower on the role physical (− 3.72), bodily pain (− 3.66), and vitality (− 2.32) scales as well as the physical health component (− 3.52). Compared to the 2009 United States general population, and after adjusting for age, gender, and differential item function, persons in Saudi Arabia had lower average scores for the physical function (− 3.10), role physical (− 4.75), social function (− 4.23), role emotional (− 5.67), and mental health (− 4.82) scales, as well as the mental health component (− 4.57). Conclusion: This Saudi normative study of patient reported outcomes supported the validity and reliability of the new Saudi SF-36v2 and found cross-national differences with the USA. [ABSTRACT FROM AUTHOR]

Aims: Reliabilities of the work environment questionnaire Copenhagen Psychosocial Questionnaire (COPSOQ) have previously been estimated by Cronbach's alpha, but since the internal consistency assumption may not apply to all COPSOQ scales, Cronbach's alpha may underestimate true reliability. This study aims to evaluate reliability in a test--retest design. Methods: We analyzed postal questionnaire data from 349 persons (of whom 283 were employees) who completed two forms with a median interval of 22 (range 6-65) days between baseline and follow-up. Test--retest reliabilities were estimated by the intraclass correlation (ICC). For scales where the internal consistency assumption was theoretically plausible, reliabilities were also estimated by Cronbach's alpha and by Green's test--retest alpha. Results: With one exception, the ICC estimated reliabilities of the COPSOQ scales were adequate or good (range 0.70--0.89). A scale concerning mutual trust between employees had a low reliability of 0.64. Among the scales where the internal consistency assumption was plausible, Cronbach's alpha was adequate or good (0.75--0.85) for seven out of eight scales. Green's retest alpha was adequate or good for six out of eight scales (0.72--0.81). Conclusions: Standard criteria for acceptable intraclass correlation reliability were achieved for all COPSOQ scales but one. The test--retest design and intraclass correlation appears to be more appropriate than Cronbach's alpha for assessing the reliability of psychosocial work environment scales. [ABSTRACT FROM AUTHOR]

Aims: The aim of the present paper is to present the development of the second version of the Copenhagen Psychosocial Questionnaire (COPSOQ II). Methods: The development of COPSOQ II took place in five main steps: (1) We considered practical experience from the use of COPSOQ I, in particular feedback from workplace studies where the questionnaire had been used; (2) All scales concerning workplace factors in COPSOQ I were analyzed for differential item functioning (DIF) with regard to gender, age and occupational status; (3) A test version of COPSOQ II including new scales and items was developed and tested in a representative sample of working Danes between 20 and 59 years of age. In all, 3,517 Danish employees participated in the study. The overall response rate was 60.4%; (4) Based on psychometric analyses, the final questionnaire was developed; and (5) Criteria-related validity of the new scales was tested. Results: The development of COPSOQ II resulted in a questionnaire with 41 scales and 127 items. New scales on values at the workplace were introduced including scales on Trust, Justice and Social inclusiveness. Scales on Variation, Work pace, Recognition, Work-family conflicts and items on offensive behaviour were also added. New scales regarding health symptoms included: Burnout, Stress, Sleeping troubles and Depressive symptoms. In general, the new scales showed good criteria validity. All in all, 57% of the items of COPSOQ I were retained in COPSOQ II. Conclusions: The COPSOQI concept has been further developed and new validated scales have been included. [ABSTRACT FROM AUTHOR]

Background: There are increasing concerns about the appropriateness of generic preference-based measures to capture health benefits in the area of mental health.Objectives: The aim of this study is to estimate preference weights for a new measure, Recovering Quality of Life (ReQoL-10), to better capture the benefits of mental healthcare.Methods: Psychometric analyses of a larger sample of mental health service users (n = 4266) using confirmatory factor analyses and item response theory were used to derive a health state classification system and inform the selection of health states for utility assessment. A valuation survey with members of the UK public representative in terms of age, sex, and region was conducted using face-to-face interviewer administered time-trade-off with props. A series of regression models were fitted to the data and the best performing model selected for the scoring algorithm.Results: The ReQoL-Utility Index (UI) classification system comprises 6 mental health items and 1 physical health item. Sixty-four health states were valued by 305 participants. The preferred model was a random effects model, with significant and consistent coefficients and best model fit. Estimated utilities modeled for all health states ranged from -0.195 (state worse than dead) to 1 (best possible state).Conclusions: The development of the ReQoL-UI is based on a novel application of item response theory methods for generating the classification system and selecting health states for valuation. Conventional time-trade-off was used to elicit utility values that are modeled to enable the generation of QALYs for use in cost-utility analysis of mental health interventions. [ABSTRACT FROM AUTHOR]