Search

Objective: Patient-reported transition ratings are supposed to reflect the change between a previous baseline health state and a present follow-up state, but may reflect the present state to a greater extent. This so-called “present state bias” (PSB) potentially threatens the validity of transition ratings. Several criteria have been proposed to assess PSB. We examined how well these criteria perform and to which extent confirmatory factor analysis (CFA) for categorical data provides an accurate assessment of the degree of PSB. Study Design and Setting: We simulated multiple samples with baseline and follow-up item responses to a hypothetical questionnaire, and transition ratings. The samples varied with respect to various distributional characteristics and the degree of PSB. The performance of criteria proposed in the literature, and a new CFA-based criterion, were evaluated by the proportion of explained variance in PSB. In addition, four real datasets were analyzed. Results: The known criteria explained 36–74% of the variance in PSB. A new CFA-based criterion, namely the ratio of the factor loadings of the transition ratings plus one, explained 81–98% of the variance in PSB across the samples. Conclusion: Present state bias in transition ratings can be estimated accurately using CFA.

Objectives Clinical practice guidelines recommend questionnaires with short recall. We compare responsiveness of ecological momentary assessments (EMAs) and retrospective assessments of thyroid-related quality of life. Study Design and Setting Patients with newly diagnosed thyrotoxicosis completed retrospective Thyroid-related Patient-Reported Outcome measures (ThyPROs) with 4-week and 1-week recall, respectively, and three daily EMAs for 4 weeks at time of inclusion and again after treatment (N = 115). Magnitude of change and statistical power (F-test statistics) were compared. Two designs were applied to the same data: Design 1 mimicked the practical realities of clinical trials by comparing 4-week recall ThyPRO administered at time of inclusion with EMA initiated at time of inclusion and collected prospectively for 1 week, thus not covering the same time frame or duration. Design 2 compared assessments covering the same 4 weeks after inclusion. Results Design 1: the estimated change and statistical power were significantly larger for 4-week ThyPRO compared with EMAs. Design 2: retrospective assessments and EMAs had comparable change and power. Repeated 1-week ThyPRO administrations increased the statistical power. Conclusion Selecting the optimal time frame for evaluation proved crucial for responsiveness. EMAs did not provide higher responsiveness than retrospective measures in either design. Repeated 1-week ThyPRO administrations increased statistical power.

Objectives To test the impact of the method of administration (MOA) on score level, reliability, and validity of scales developed in the Patient Reported Outcomes Measurement Information System (PROMIS). Study Design and Setting Two nonoverlapping parallel forms each containing eight items from each of three PROMIS item banks (Physical Function, Fatigue, and Depression) were completed by 923 adults with chronic obstructive pulmonary disease, depression, or rheumatoid arthritis. In a randomized crossover design, subjects answered one form by interactive voice response (IVR) technology, paper questionnaire (PQ), personal digital assistant (PDA), or personal computer (PC) and a second form by PC, in the same administration. Method equivalence was evaluated through analyses of difference scores, intraclass correlations (ICCs), and convergent/discriminant validity. Results In difference score analyses, no significant mode differences were found and all confidence intervals were within the prespecified minimal important difference of 0.2 standard deviation. Parallel-forms reliabilities were very high (ICC = 0.85–0.93). Only one across-mode ICC was significantly lower than the same-mode ICC. Tests of validity showed no differential effect by MOA. Participants preferred screen interface over PQ and IVR. Conclusion We found no statistically or clinically significant differences in score levels or psychometric properties of IVR, PQ, or PDA administration compared with PC.

Objective: The Patient-Reported Outcomes Measurement Information System (PROMIS) was initiated to improve precision, reduce respondent burden, and enhance the comparability of health outcomes measures. We used item response theory (IRT) to construct and evaluate a preliminary item bank for physical function assuming four subdomains. Study Design and Setting: Data from seven samples (N 517,726) using 136 items from nine questionnaires were evaluated. A generalized partial credit model was used to estimate item parameters, which were normed to a mean of 50 (SD 510) in the US population. Item bank properties were evaluated through Computerized Adaptive Test (CAT) simulations. Results: IRT requirements were fulfilled by 70 items covering activities of daily living, lower extremity, and central body functions. The original item context partly affected parameter stability. Items on upper body function, and need for aid or devices did not fit the IRT model. In simulations, a 10-item CAT eliminated floor and decreased ceiling effects, achieving a small standard error (!2.2) across scores from 20 to 50 (reliability O0.95 for a representative US sample). This precision was not achieved over a similar range by any comparable fixed length item sets. Conclusion: The methods of the PROMIS project are likely to substantially improve measures of physical function and to increase the

Background and Objective The goal was to develop a shortened version of the EORTC QLQ-C30 for use in palliative care. We wanted to keep as few items as possible in each scale while still being able to compare results with studies using the original scales. We examined the possibilities of shortening the physical functioning, cognitive functioning, fatigue, and nausea and vomiting scales. Study Design and Setting The shortening was based on 2,366 (physical functioning) and 10,815 (three other scales) observations, respectively. We used item response theory to construct scoring algorithms for predicting scores on the original scales. Results Evaluations showed that a three-item physical scale, a two-item fatigue scale, and a one-item nausea or vomiting scale predicted the scores on the original scales with excellent agreement and had measurement abilities similar to the original scales with no loss or only a little loss in power to detect group differences. The results of the cognitive functioning scale indicated problems when predicting scores from a shortened version. Conclusion Given the favorable results for the physical functioning, fatigue, and nausea or vomiting scales we expect that the shortened versions of these scales will be included in the abbreviated version of the EORTC QLQ-C30 for palliative care.

Breast cancer and its treatment have been associated with psychological morbidity. In this study our aim was to quantify the excess anxiety and depression resulting from breast cancer. We compared 538 newly diagnosed breast cancer patients at low risk of recurrence (87.0% responded) to 872 women randomly selected from the Danish general population (69.7% responded) using the Hospital Anxiety and Depression Scale (HADS). Contrary to expectations, the proportions classified as “cases” of anxiety and depression were not significantly different in the two groups. The breast cancer patients’ mean HADS scores were significantly lower than those in the general population sample (anxiety, P = 0.021; depression, P

Rasch models for polytomous items were used to assess the scaling assumptions and compare item response patterns in the 10-item SF-36 physical functioning scale (PF-10) for general population respondents in Denmark, Germany, Italy, the Netherlands, Sweden, the United Kingdom, and the United States. The Rasch model of physical functioning developed in the United States was compared to models for other countries, and each country was compared to a multinational composite. Strong scale congruence across the seven countries was demonstrated; items that varied between countries and from the composite may reflect unique cultural response patterns or differences in translation. Scoring algorithms based on the Rasch model for each country were superior to the current Likert scoring in tests of relative validity (RV) in discriminating among age groups in all countries. In relation to the Likert PF-10 scoring (RV = 1.00), scores estimated using the Rasch rating scale model achieve a median RV of 1.31 (range: 1.01-1.59), while the Rasch partial credit model attained a median RV of 1.44 (range: 1.01-2.23). Rasch models hold good potential for improving health status measures, estimating individual scores when responses to scale items are missing, and equating scores across countries.

Data from general population samples in 11 countries (n = 1483 to 9151) were used to assess data quality and test the assumptions underlying the construction and scoring of multi-item scales from the SF-36 Health Survey. Across all countries, the rate of item-level missing data generally was low, although slightly higher for items printed in the grid format. In each country, item means generally were clustered as hypothesized within scales. Correlations between items and hypothesized scales were greater than 0.40 with one exception, supporting item internal consistency. Items generally correlated significantly higher with their own scale than with competing scales, supporting item discriminant validity. Scales could be constructed for 93-100% of respondents. Internal consistency reliability of the eight SF-36 scales was above 0.70 for all scales, with two exceptions. Floor effects were low for all except the two role functioning scales; ceiling effects were high for both role functioning scales and also were noteworthy for the Physical Functioning, Bodily Pain, and Social Functioning scales in some countries. These results support the construction and scoring of the SF-36 translations in these 11 countries using the method of summated ratings.

Increasingly, translated and culturally adapted health-related quality of life measures are being used in cross-cultural research. To assess comparability of results, researchers need to know the comparability of the content of the questionnaires used in different countries. Based on an item-by-item discussion among International Quality of Life Assessment (IQOLA) investigators of the content of the translated versions of the SF-36 in 10 countries, we discuss the difficulties that arose in translating the SF-36. We also review the solutions identified by IQOLA investigators to translate items and response choices so that they are appropriate within each country as well as comparable across countries. We relate problems and solutions to ratings of difficulty and conceptual equivalence for each item. The most difficult items to translate were physical functioning items that refer to activities not common outside the United States and items that use colloquial expressions in the source version. Identifying the origin of the source items, their meaning to American English-speaking respondents and American English synonyms, in response to country-specific translation issues, greatly helped the translation process. This comparison of the content of translated SF-36 items suggests that the translations are culturally appropriate and comparable in their content.

A crucial prerequisite to the use of the SF-36 Health Survey in multinational studies is the reproduction of the conceptual model underlying its scoring and interpretation. Structural equation modeling (SEM) was used to test these aspects of the construct validity of the SF-36 in ten IQOLA countries: Denmark, France, Germany, Italy, the Netherlands, Norway, Spain, Sweden, the United Kingdom, and the United States. Data came from general population surveys fielded to gather normative data. Measurement and structural models developed in the United States were cross-validated in random halves of the sample in each country. SEM analyses supported the eight first-order factor model of health that underlies the scoring of SF-36 scales and two second-order factors that are the basis for summary physical and mental health measures. A single third-order factor was also observed in support of the hypothesis that all responses to the SF-36 are generated by a single, underlying construct--health. In addition, a third second-order factors, interpreted as general well-being, was shown to improve the fit of the model. This model (including eight first-order factors, three second-order factors, and one third-order factor) was cross-validated using a holdout sample within the United States and in each of the nine other countries. These results confirm the hypothesized relationships between SF-36 items and scales and justify their scoring in each country using standard algorithms. Results also suggest that SF-36 scales and summary physical and mental health measures will have similar interpretations across countries. The practical implications of a third second-order SF-36 factor (general well-being) warrant further study.

The similarity in meaning assigned to response choice labels from the SF-36 Health Survey (SF-36) was evaluated across countries. Convenience samples of judges (range, 10 to 117; median = 48) from 13 countries rated translations of response choice labels, using a variation of the Thurstone method of equal appearing intervals. Judges marked a point on a 10-cm line-representing the magnitude of a response choice label (e.g., "good" relative to the anchors of "poor" and "excellent"). Ratings were evaluated to determine the ordinal consistency of response choice labels within a response scale; the degree to which differences between adjacent response choice labels were equal interval; and the amount of variance due to response choice label, country, judge, and interaction between response choice label and country. Results confirmed the hypothesized ordering of response choice labels; the percentage of ordinal pairs ranged from 88.7% to 100% (median = 98.2%) across countries and response scales. Examination of the average magnitudes of response choice labels supported the "quasi-interval" nature of the scales. Analysis of variance (ANOVA) results supported the generalizability of response choice magnitudes across countries; labels explained 64% to 77% of the variance in ratings, and country explained 1% to 3%. These results support the equivalence of SF-36 response choice labels across countries. Departures from the assumption of equal intervals, when observed, were similar across countries and were greatest for the two response scales that are recalibrated under standard SF-36 scoring. Results provide justification for scoring translations of individual items using standard SF-36 scoring; whether these items form the same scales in other countries as they do in the United States is evaluated with tests of scaling assumptions.

This article reports on the Danish translation of SF-36 and discusses the procedures used for translation improvement, translation evaluation, and scale evaluation. We followed the standard procedures of the International Quality of Life Assessment (IQOLA) Project including forward and backward translation, independent assessment of translation quality, assessment of response-choice weighting through visual analogue scale (VAS) investigations, and psychometric testing of the translated questionnaire. We found that backward translation, independent quality assessment, and VAS studies provided useful information for translation improvement. The Danish SF-36 received a favorable translation evaluation by independent rating; however, interrater agreement was low. Preliminary validity studies generally supported the internal consistency and homogeneity of the Danish SF-36, and the questionnaire performed satisfactorily in distinguishing depressive patients from nonpatients. On the basis of this and other studies, we recommend use of the Danish SF-36 in research.

Data from general population surveys (n = 1483 to 9151) in nine European countries (Denmark, France, Germany, Italy, the Netherlands, Norway, Spain, Sweden, and the United Kingdom) were analyzed to cross-validate the selection of questionnaire items for the SF-12 Health Survey and scoring algorithms for 12-item physical and mental component summary measures. In each country, multiple regression methods were used to select 12 SF-36 items that best reproduced the physical and mental health summary scores for the SF-36 Health Survey. Summary scores then were estimated with 12 items in three ways: using standard (U.S.-derived) SF-12 items and scoring algorithms; standard items and country-specific scoring; and country-specific sets of 12 items and scoring. Replication of the 36-item summary measures by the 12-item summary measures was then evaluated through comparison of mean scores and the strength of product-moment correlations. Product-moment correlations between SF-36 summary measures and SF-12 summary measures (standard and country-specific) were very high, ranging from 0.94-0.96 and 0.94-0.97 for the physical and mental summary measures, respectively. Mean 36-item summary measures and comparable 12-item summary measures were within 0.0 to 1.5 points (median = 0.5 points) in each country and were comparable across age groups. Because of the high degree of correspondence between summary physical and mental health measures estimated using the SF-12 and SF-36, it appears that the SF-12 will prove to be a practical alternative to the SF-36 in these countries, for purposes of large group comparisons in which the focus is on overall physical and mental health outcomes.

Statistical analyses of Differential Item Functioning (DIF) can be used for rigorous translation evaluations. DIF techniques test whether each item functions in the same way, irrespective of the country, language, or culture of the respondents. For a given level of health, the score on any item should be independent of nationality. This requirement can be tested through contingency-table methods, which are efficient for analyzing all types of items. We investigated DIF in the Danish translation of the SF-36 Health Survey, using two general population samples (USA, n = 1,506; Denmark, n = 3,950). DIF was identified for 12 out of 35 items. These results agreed with independent ratings of translation quality, but the statistical techniques were more sensitive. When included in scales, the items exhibiting DIF had only a little impact on conclusions about cross-national differences in health in the general population. However, if used as single items, the DIF items could seriously bias results from cross-national comparisons. Also, the DIF items might have larger impact on cross-national comparison of groups with poorer health status. We conclude that analysis of DIF is useful for evaluating questionnaire translations.

Item bias (differential item functioning) analysis examines whether the construction of an index from two or more variables results in bias in relation to sex, age, or other criteria. Item bias may lead to erroneous conclusions because of distortion or dilution of the effects measured. In comparing groups, item bias analysis, tests whether the information about possible differences between groups, obtained by the variables constituting an index, are correctly passed on by the index score. We examined a quality of life questionnaire answered by 1189 breast cancer patients. We found age-bias or bias in the comparison of groups receiving different treatments in three out of nine indexes. Recommendations for the interpretation of these indexes are made. Item bias analysis is a useful method examining an issue not covered by traditional psychometric tests.

Objectives To provide a standardized metric for the assessment of depression severity to enable comparability among results of established depression measures. Study Design and Setting A common metric for 11 depression questionnaires was developed applying item response theory (IRT) methods. Data of 33,844 adults were used for secondary analysis including routine assessments of 23,817 in- and outpatients with mental and/or medical conditions (46% with depressive disorders) and a general population sample of 10,027 randomly selected participants from three representative German household surveys. Results A standardized metric for depression severity was defined by 143 items, and scores were normed to a general population mean of 50 (standard deviation = 10) for easy interpretability. It covers the entire range of depression severity assessed by established instruments. The metric allows comparisons among included measures. Large differences were found in their measurement precision and range, providing a rationale for instrument selection. Published scale-specific threshold scores of depression severity showed remarkable consistencies across different questionnaires. Conclusion An IRT-based instrument-independent metric for depression severity enables direct comparisons among established measures. The "common ruler" simplifies the interpretation of depression assessment by identifying key thresholds for clinical and epidemiologic decision making and facilitates integrative psychometric research across studies, including meta-analysis.

Objective To document the development and psychometric evaluation of the Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function (PF) item bank and static instruments. Study Design and Setting The items were evaluated using qualitative and quantitative methods. A total of 16,065 adults answered item subsets ( n >2,200/item) on the Internet, with oversampling of the chronically ill. Classical test and item response theory methods were used to evaluate 149 PROMIS PF items plus 10 Short Form-36 and 20 Health Assessment Questionnaire-Disability Index items. A graded response model was used to estimate item parameters, which were normed to a mean of 50 (standard deviation [SD]=10) in a US general population sample. Results The final bank consists of 124 PROMIS items covering upper, central, and lower extremity functions and instrumental activities of daily living. In simulations, a 10-item computerized adaptive test (CAT) eliminated floor and decreased ceiling effects, achieving higher measurement precision than any comparable length static tool across four SDs of the measurement range. Improved psychometric properties were transferred to the CAT's superior ability to identify differences between age and disease groups. Conclusion The item bank provides a common metric and can improve the measurement of PF by facilitating the standardization of patient-reported outcome measures and implementation of CATs for more efficient PF assessments over a larger range.

Background and objective This study examined the psychometric properties and equivalence of the six-item Headache Impact Test (HIT-6) across 11 languages in 14 countries. Methods A multicenter, international cross-sectional study conducted in a primary care setting. Data obtained from 1,171 adults from 14 countries who consulted their primary care physician for headache completed the HIT-6 questionnaire and a headache survey were included in this analysis. Item-level statistics (e.g., range of response choices used by participants), item–scale statistics (e.g., item–total correlations), scale level statistics (e.g., internal consistency reliability), and tests of differential item functioning were conducted to examine the psychometric properties of all HIT-6 translations and their comparability across translations. Results Across languages, missing data were low, item–scale correlations were high, reliability was adequate, and item-level statistics were generally comparable. We found only minor differential item functioning, suggesting that the HIT-6 translations are equivalent to the U.S. English form. Conclusions Psychometric analyses indicate that most HIT-6 translations (Canadian English, French, Greek, Hungarian, UK English, Hebrew, Portuguese, German, Spanish, and Dutch) are comparable to U.S. English. Improvements may be needed in the Finnish and Slovakian translations and the appropriateness of using the HIT-6 in South Africa should be explored further.

Evaluate a patient-reported outcomes questionnaire that uses computerized adaptive testing (CAT) to measure the impact of osteoarthritis (OA) on functioning and well-being.OA patients completed 37 questions about the impact of OA on physical, social and role functioning, emotional well-being, and vitality. Questionnaire responses were calibrated and scored using item response theory, and two scores were estimated: a Total-OA score based on patients' responses to all 37 questions, and a simulated CAT-OA score where the computer selected and scored the five most informative questions for each patient. Agreement between Total-OA and CAT-OA scores was assessed using correlations. Discriminant validity of Total-OA and CAT-OA scores was assessed with analysis of variance. Criterion measures included OA pain and severity, patient global assessment, and missed work days.Simulated CAT-OA and Total-OA scores correlated highly (r = 0.96). Both Total-OA and simulated CAT-OA scores discriminated significantly between patients differing on the criterion measures. F-statistics across criterion measures ranged from 39.0 (P.001) to 225.1 (P.001) for the Total-OA score, and from 40.5 (P.001) to 221.5 (P.001) for the simulated CAT-OA score.CAT methods produce valid and precise estimates of the impact of OA on functioning and well-being with significant reduction in response burden.

We used general population data (n = 4084) to examine data completeness, response consistency, tests of scaling assumptions, and reliability of the Danish SF-36 Health Survey. We compared traditional multitrait scaling analyses to analyses using polychoric correlations and Spearman correlations. The frequency of missing values was low, except for elderly people and people with lower levels of education. Response consistency was high and compared well with results for the U.S. SF-36. For respondents with computable scales in all eight domains, scaling assumptions (item internal consistency, item discriminant validity, equal item-own scale correlations, and equal variances) were satisfactory in the total sample and in all subgroups. The SF-36 could discriminate between levels of health in all subgroups, but there were skewness, kurtosis, and ceiling effects in many subgroups (elderly people and people with chronic diseases excepted). Concerning correlation methods, we found interesting differences indicating advantages of using methods that do not assume a normal distribution of answers as an addition to traditional methods.

Objectives This study aimed to develop and evaluate a first computerized adaptive test (CAT) for the measurement of stress perception (Stress-CAT), in terms of the two dimensions: exposure to stress and stress reaction . Study Design and Setting Item response theory modeling was performed using a two-parameter model (Generalized Partial Credit Model). The evaluation of the Stress-CAT comprised a simulation study and real clinical application. A total of 1,092 psychosomatic patients (N1) were studied. Two hundred simulees (N2) were generated for a simulated response data set. Then the Stress-CAT was given to n =116 inpatients, (N3) together with established stress questionnaires as validity criteria. Results The final banks included n =38 stress exposure items and n =31 stress reaction items. In the first simulation study, CAT scores could be estimated with a high measurement precision (SE 0.90) using 7.0±2.3 (M±SD) stress reaction items and 11.6±1.7 stress exposure items. The second simulation study reanalyzed real patients data (N1) and showed an average use of items of 5.6±2.1 for the dimension stress reaction and 10.0±4.9 for the dimension stress exposure . Convergent validity showed significantly high correlations. Conclusions The Stress-CAT is short and precise, potentially lowering the response burden of patients in clinical decision making.