Search

Objective: To estimate the reliability and validity of survey measures used to evaluate health plans and providers from the consumer's perspective.Data Sources: Members (166,074) of 306 U.S. health plans obtained from the National CAHPS Benchmarking Database 2.0, a voluntary effort in which sponsors of CAHPS surveys contribute data to a common repository.Study Design: Members of privately insured health plans serving public and private employers across the United States were surveyed by mail and telephone. Interitem correlations and correlations of items with the composite scores were estimated. Plan-level and internal consistency reliability are estimated. Multivariate associations of composite measures with global ratings are also examined to assess construct validity. Confirmatory factor analysis is used to examine the factor structure of the measure.Findings: Plan-level reliability of all CAHPS 2.0 reporting composites is high with the given sample sizes. Fewer than 170 responses per plan would achieve plan-level reliability of .70 for the five composites. Two of the composites display high internal consistency (Cronbach's alpha > or = .75), while responses to items in the other three composites were not as internally consistent (Cronbach's alpha from .58 to .62). A five-factor model representing the CAHPS 2.0 composites fits the data better than alternative two- and three-factor models.Conclusion: Two of the five CAHPS 2.0 reporting composites have high internal consistency and plan-level reliability. The other three summary measures were reliable at the plan level and approach acceptable levels of internal consistency. Some of the items that form the CAHPS 2.0 adult core survey, such as the measure of waiting times in the doctor's office, could be improved. The five-dimension model of consumer assessments best fits the data among the privately insured; therefore, consumer reports using CAHPS surveys should provide feedback using five composites. [ABSTRACT FROM AUTHOR]

Objective: To assess health-related quality of life (HRQOL) in men treated for clinically localized prostate cancer.Design: A cross-sectional analysis of HRQOL after treatment with radical prostatectomy, pelvic irradiation, or observation alone for clinically localized prostatic adenocarcinoma, and in age-matched comparison patients.Setting: A large managed care population in California.Subjects: A total of 528 men, including 214 treated for clinically localized prostate cancer (41 with evidence of metastatic disease were excluded from this analysis) and 273 age-matched, ZIP code-matched comparison patients without prostate cancer. Cancer patients were analyzed in three treatment groups: radical prostatectomy (n = 98), primary pelvic irradiation (n = 56), and observation alone (n = 60).Main Outcome Measures: General HRQOL was measured with the RAND 36-Item Health Survey 1.0. Cancer-specific HRQOL was measured with the CAncer Rehabilitation Evaluation System-Short Form and the Functional Assessment of Cancer Therapy-General form. Disease-targeted quality of life was measured with a new instrument assessing function and bother in three organ systems: sexual, urinary, and bowel.Results: No differences among treatment groups were seen in comparisons of general HRQOL: Significant differences among treatment groups were seen in both function and bother in the prostate-targeted measures of sexual, urinary, and bowel domains. When cancer patients were compared with men of similar age without prostate cancer, differences were seen in the sexual, urinary, and bowel function and bother but not in general HRQOL measures. Although cancer-free men were found not to have full potency or continence, prostate cancer patients treated with surgery or radiation reported significantly worse sexual, urinary, and bowel function than men without cancer. Men who had undergone nerve-sparing prostatectomy did not differ from those who had undergone standard prostatectomy, but the power to detect a difference was low.Conclusions: Although no differences were seen in general HRQOL, three disease-targeted domains were found to differ significantly among the treatment groups and comparison patients. Even after controlling for the sexual and urinary dysfunction experienced by older men without cancer, those receiving therapeutic interventions for their prostate cancer were found to have poorer disease-targeted HRQOL: We conclude that in addition to general HRQOL, disease-targeted measures must be used to assess outcomes of care in men treated for localized prostate cancer. [ABSTRACT FROM AUTHOR]

Purpose: To assess diagnostic accuracy of two self-administered depression measures compared to an interviewer-administered measure in subjects with Parkinson's disease (PD), and to analyze clinical and sociodemographic factors associated with disagreement among the three depression assessment tools.Methods: We assessed 214 PD subjects using the Patient Health Questionnaire-9 (PHQ-9), the Geriatric Depression Scale-15 (GDS-15), and the Structured Clinical Interview for the DSM-IV depression module (SCID). Diagnostic accuracy of the PHQ-9 and GDS-15 compared to the SCID was evaluated. Multivariate logistic regression was conducted to analyze factors associated with measure disagreement. We compared item agreement between the PHQ-9 and SCID to test the hypothesis that there would be less agreement between items assessing depression symptoms overlapping with common PD symptoms, compared to items having minimal overlap with PD manifestations.Results: Compared to SCID diagnosis of major depression, PHQ-9 sensitivity is 50% and specificity is 93%; GDS-15 sensitivity is 43% and specificity is 96%. The GDS-15 has 85% sensitivity and 79% specificity and the PHQ-9 has 54% sensitivity and 85% specificity compared to SCID diagnosis of minor or major depression. The PHQ-9 and SCID show more agreement on items unrelated to PD manifestations. Pain was the only factor associated with disagreement between the SCID and PHQ-9.Conclusion: Compared to the PHQ-9, the GDS-15 had higher sensitivity and similar positive predictive value, suggesting it is a superior screening tool in clinical applications for PD. On future depression screening or diagnostic instruments, consideration should be given to excluding depression items overlapping with PD manifestations. [ABSTRACT FROM AUTHOR]

Context: The Centers for Medicare and Medicaid Services will introduce the reporting of patient surveys in 2008. The Consumer Assessment of Health Care Providers and Systems (CAHPS) Hospital Survey contains 18 questions about hospital care. Internal consistency reliability of the discharge information scale is relatively low and some important domains of care are not represented.Objective: To determine whether adding questions increases the reliability and validity of the survey.Data Sources and Study Setting: Surveys of patients at 181 hospitals participating in the California Hospitals Assessment and Reporting Taskforce (CHART), an initiative for voluntary public reporting of hospital performance in California.Study Design: CHART added nine questions to the CAHPS Hospital Survey; two to improve reliability of the discharge information domain, five to create a coordination of care domain, and two relating to interpreter services.Data Collection: Surveys were sent to randomly selected patients from each CHART hospital.Principal Findings: A total of 40,172 surveys were included. Adding the new discharge information questions improved the internal consistency reliability from 0.45 to 0.72 and the hospital-level reliability from 0.75 to 0.81. New coordination of care composites had good internal consistency reliabilities ranging from 0.58 to 0.70 and hospital-level reliabilities ranging from 0.84 to 0.87. The new coordination of care composites were more closely correlated with overall hospital ratings and willingness to recommend than six of the seven original domains.Conclusions: The additional discharge information questions and the new coordination of care questions significantly improved the psychometric properties of the CAHPS Hospital Survey. [ABSTRACT FROM AUTHOR]

Objectives: To estimate the associations among hospital-level scores from the Consumer Assessments of Healthcare Providers and Systems (CAHPS) Hospital pilot survey within and across different services (surgery, obstetrics, medical), and to evaluate differences between hospital- and patient-level analyses.Data Source: CAHPS Hospital pilot survey data provided by the Centers for Medicare and Medicaid Services.Study Design: Responses to 33 questionnaire items were analyzed using patient- and hospital-level exploratory factor analytic (EFA) methods to identify both a patient-level and hospital-level composite structures for the CAHPS Hospital survey. The latter EFA was corrected for patient-level sampling variability using a hierarchical model. We compared results of these analyses with each other and to separate EFAs conducted at the service level. To quantify the similarity of assessments across services, we compared correlations of different composites within the same service with those of the same composite across different services.Data Collection: Cross-sectional data were collected during the summer of 2003 via mail and telephone from 19,720 patients discharged from November 2002 through January 2003 from 132 hospitals in three states.Principal Findings: Six factors provided the best description of inter-item covariation at the patient level. Analyses that assessed variability across both services and hospitals suggested that three dimensions provide a parsimonious summary of inter-item covariation at the hospital level. Hospital-level factor structures also differed across services; as much variation in quality reports was explained by service as by composite.Conclusions: Variability of CAHPS scores across hospitals can be reported parsimoniously using a limited number of composites. There is at least as much distinct information in composite scores from different services as in different composite scores within each service. Because items cluster slightly differently in the different services, service-specific composites may be more informative when comparing patients in a given service across hospitals. When studying individual-level variability, a more differentiated structure is probably more appropriate. [ABSTRACT FROM AUTHOR]

Objective: To identify a parsimonious subset of reliable, valid, and consumer-salient items from 33 questions asking for patient reports about hospital care quality.Data Source: CAHPS Hospital Survey pilot data were collected during the summer of 2003 using mail and telephone from 19,720 patients who had been treated in 132 hospitals in three states and discharged from November 2002 to January 2003.Methods: Standard psychometric methods were used to assess the reliability (internal consistency reliability and hospital-level reliability) and construct validity (exploratory and confirmatory factor analyses, strength of relationship to overall rating of hospital) of the 33 report items. The best subset of items from among the 33 was selected based on their statistical properties in conjunction with the importance assigned to each item by participants in 14 focus groups.Principal Findings: Confirmatory factor analysis (CFA) indicated that a subset of 16 questions proposed to measure seven aspects of hospital care (communication with nurses, communication with doctors, responsiveness to patient needs, physical environment, pain control, communication about medication, and discharge information) demonstrated excellent fit to the data. Scales in each of these areas had acceptable levels of reliability to discriminate among hospitals and internal consistency reliability estimates comparable with previously developed CAHPS instruments.Conclusion: Although half the length of the original, the shorter CAHPS hospital survey demonstrates promising measurement properties, identifies variations in care among hospitals, and deals with aspects of the hospital stay that are important to patients' evaluations of care quality. [ABSTRACT FROM AUTHOR]

Background: On-line physician education is increasing, but its efficacy in comparison with existing self-study methods is unknown.Objective: To compare knowledge, learning efficiency, and learner satisfaction produced by self-study of World Wide Web-based and print-based guidelines for care after acute myocardial infarction.Design: Randomized, controlled trial.Setting: 12 family medicine and internal medicine residency programs at four universities.Participants: 162 residents.Interventions: In proctored sessions, participants were randomly assigned to study from printed materials or from SAGE (Self-Study Acceleration with Graphic Evidence), a Web-based tutorial system. Both methods used identical self-assessment questions and answers and guideline text, but SAGE featured hyperlinks to specific guideline passages and graphic evidence animations.Measurements: Scores on multiple-choice knowledge tests, score gain per unit of study time, and ratings on a learner satisfaction scale.Results: Immediate post-test scores on a 20-point scale were similar in the SAGE and control groups (median score, 15.0 compared with 14.5; P>0.2), but SAGE users spent less time studying (median, 27.0 compared with 38.5 minutes; P<0.001) and therefore had greater learning efficiency (median score gain, 8.6 compared with 6.7 points per hour; P = 0.04). On a scale of 5 to 20, SAGE users were more satisfied with learning (median rating, 17.0 compared with 15.0; P<0.001). After 4 to 6 months, knowledge had decreased to the same extent in the SAGE and control groups (median score, 12.0 compared with 11.0; P = 0.12).Conclusions: On-line tutorials may produce greater learning efficiency and satisfaction than print materials do, but one self-study exposure may be insufficient for long-term knowledge retention. Further research is needed to identify instructional features that motivate greater final learning and retention. [ABSTRACT FROM AUTHOR]

Purpose: The purpose of this study was to evaluate the reliability and validity of the Thai Quality of Life in Children (ThQLC) and compare it with the Pediatric Quality of Life Inventory (PedsQL™ 4.0) in a sample of children receiving long-term HIV care in Thailand.Methods: The ThQLC and the PedsQL™ 4.0 were administered to 292 children with HIV infection aged 8-16 years. Clinical parameters such as the current viral load, CD4 percent, and clinical staging were obtained by medical record review.Results: Three out of five ThQLC scales and three out of four PedsQL™ 4.0 scales had acceptable internal consistency reliability (i.e., Cronbach's alpha >0.70). Cronbach's alpha values of each scale ranged from 0.52 to 0.75 and 0.57 to 0.75 for the ThQLC and the PedsQL™ 4.0, respectively. Corresponding scales (physical functioning, emotional well-being, social functioning, and school functioning) of the ThQLC and the PedsQL™ 4.0 correlated substantially with one another (r = 0.47, 0.67, 0.59 and 0.56, respectively). Both ThQLC and PedsQL™ 4.0 overall scores significantly correlated with the child's self-rated severity of the illness (r = -0.23 for the ThQLC and -0.28 for the PedsQL™ 4.0) and the caregiver's rated overall quality of life (r = 0.07 for the ThQLC and 0.13 for the PedsQL™ 4.0). The overall score of the ThQLC correlated with clinical and immunologic categories of the United State-Centers for Disease Control and Prevention (US-CDC) classification system (r = -0.12), while the overall score of the PedsQL™ 4.0 significantly correlated with the number of disability days (r = -0.12) and CD4 percent (r = -0.15). However, the overall score from both instruments were not significantly different by clinical stages of HIV disease. A multitrait-multimethod analysis results demonstrated that the average convergent validity and off-diagonal correlations were 0.58 and 0.45, respectively. Discriminant validity was partially supported with 62% of validity diagonal correlations exceeding correlations between different domains (discriminant validity successes). The Hays-Hayashi MTMM quality index was 0.61. Multivariate regression analysis revealed that the ThQLC physical functioning scale provided unique information in predicting child self-rated severity of the illness and overall quality of life beyond that explained by the PedsQL™ 4.0 in Thai children with HIV infection.Conclusions: We found evidence in support of the reliability and validity of the ThQLC and the PedsQL™ 4.0 for measuring the health-related quality of life of Thai children with HIV infection. [ABSTRACT FROM AUTHOR]

The proportion of minority elders with dementia in the United States is projected to rise substantially. This study elicited perceptions of the caregiving experience from informal caregivers of persons with dementia, across different ethnicities. Six focus groups with 47 dementia caregivers of African-American, Chinese-American, EuroAmerican, and Hispanic-American ethnicities were conducted. Caregiving roles, concern about the person with dementia, and unmet information and resource needs were expressed similarly. However, perspectives differed across ethnic groups on stigma surrounding dementia, benefits of caregiving, spirituality/religion to ease caregiving burden, and language barriers and discrimination. Findings suggest that interventions to reduce disparities in dementia care quality need to address ethnic variations in caregiving experiences. [ABSTRACT FROM AUTHOR]

Objective: To describe translation and cultural adaptation procedures, and examine the degree of equivalence between the Spanish and English versions of the Agency for Healthcare Research and Quality's (AHRQ) Consumer Assessments of Healthcare Providers and Systems (CAHPS) Hospital Survey (H-CAHPS) of patient experiences with care.Data Sources: Cognitive interviews on survey comprehension with 12 Spanish-speaking and 31 English-speaking subjects. Psychometric analyses of 586 responses to the Spanish version and 19,134 responses to the English version of the H-CAHPS survey tested in Arizona, Maryland, and New York in 2003.Study Design: A forward/backward translation procedure followed by committee review and cognitive testing was used to ensure a translation that was both culturally and linguistically appropriate. Responses to the two language versions were compared to evaluate equivalence and assess the reliability and validity of both versions.Data Collection/extraction Methods: Comparative analyses were carried out on the 32 items of the shortened survey version, focusing on 16 items that comprise seven composites representing different aspects of hospital care quality (communication with nurses, communication with doctors, communication about medicines, nursing services, discharge information, pain control, and physical environment); three items that rate the quality of the nursing staff, physician staff, and the hospital overall; one item on intention to recommend the hospital. The other 12 items used in the analyses addressed mainly respondent characteristics. Analyses included item descriptives, correlations, internal consistency reliability of composites, factor analysis, and regression analysis to examine construct validity.Principal Findings: Responses to both language versions exhibit similar patterns with respect to item-scale correlations, factor structure, content validity, and the association between each of the seven qualities of care composites with both the hospital rating and intention to recommend the hospital. Internal consistency reliability was slightly, yet significantly lower for the Spanish-language respondents for five of the seven composites, but overall the composites were generally equivalent across language versions.Conclusions: The results provide preliminary evidence of the equivalence between the Spanish and English versions of H-CAHPS. The translated Spanish version can be used to assess hospital quality of care for Spanish speakers, and compare results across these two language groups. [ABSTRACT FROM AUTHOR]

Objective: To estimate the effect of survey mode (mail versus telephone) on reports and ratings of hospital care.Data Sources/study Setting: The total sample included 20,826 patients discharged from a group of 24 distinct hospitals in three states (Arizona, Maryland, New York). We collected CAHPS data in 2003 by mail and telephone from 9,504 patients, of whom 39 percent responded by telephone and 61 percent by mail.Study Design: We estimated mode effects in an observational design, using both propensity score blocking and (ordered) logistic regression on covariates. We used variables derived from administrative data (either included as covariates in the regression function or used in estimating the propensity score) grouped in three categories: individual characteristics, characteristics of the stay and hospital, and survey administration variables.Data Collection/extraction Methods: We mailed a 66-item questionnaire to everyone in the sample and followed up by telephone with those who did not respond.Principal Findings: We found significant (p<.01) mode effects for 13 of the 21 questions examined in this study. The maximum magnitude of the survey mode effect was an 11 percentage-point difference in the probability of a "yes" response to one of the survey questions. Telephone respondents were more likely to rate care positively and health status negatively, compared with mail respondents. Standard regression-based case-mix adjustment captured much of the mode effects detected by propensity score techniques in this application.Conclusions: Telephone mode increases the propensity for more favorable evaluations of care for more than half of the items examined. This suggests that mode of administration should be standardized or carefully adjusted for. Alternatively, further item development may minimize the sensitivity of items to mode of data collection. [ABSTRACT FROM AUTHOR]

Objective: To examine the predictors of unit and item nonresponse, the magnitude of nonresponse bias, and the need for nonresponse weights in the Consumer Assessment of Health Care Providers and Systems (CAHPS) Hospital Survey.Methods: A common set of 11 administrative variables (41 degrees of freedom) was used to predict unit nonresponse and the rate of item nonresponse in multivariate models. Descriptive statistics were used to examine the impact of nonresponse on CAHPS Hospital Survey ratings and reports.Results: Unit nonresponse was highest for younger patients and patients other than non-Hispanic whites (p<.001); item nonresponse increased steadily with age (p<.001). Fourteen of 20 reports of ratings of care had significant (p<.05) but small negative correlations with nonresponse weights (median -0.06; maximum -0.09). Nonresponse weights do not improve overall precision below sample sizes of 300-1,000, and are unlikely to improve the precision of hospital comparisons. In some contexts, case-mix adjustment eliminates most observed nonresponse bias.Conclusions: Nonresponse weights should not be used for between-hospital comparisons of the CAHPS Hospital Survey, but may make small contributions to overall estimates or demographic comparisons, especially in the absence of case-mix adjustment. [ABSTRACT FROM AUTHOR]

Background: Health-related quality of life refers to how well people are able to perform daily activities (functioning) and how they feel about their lives (well-being). The relationship between symptoms and health-related quality of life has not been fully explored.Objective: To estimate the association of HIV symptoms with health-related quality of life and with disability days.Design: Prospective cohort study.Setting: HIV providers in 28 urban and 24 rural randomly selected sites throughout the United States.Patients: Nationally representative sample of 2267 adults with known HIV infection who were interviewed in 1996 and again between 1997 and 1998.Measurements: Symptoms, two single-item global measures of health-related quality of life (perceived health and perceived quality of life), and disability days.Results: White patches in the mouth; nausea or loss of appetite; persistent cough, difficulty breathing, or difficulty catching one's breath; and weight loss were associated with more disability days and worse scores on both health-related quality-of-life measures. Headache; pain in the mouth, lips, or gums; dry mouth; and sinus infection, pain, or discharge were associated with worse perceived health. Pain in the mouth, lips, or gums; trouble with eyes; pain, numbness, or tingling of hands or feet; and diarrhea or loose or watery stools were associated with worse perceived quality of life. Headache and fever, sweats, or chills were associated with more disability days.Conclusions: Several symptoms are associated with worse health-related quality of life and more disability days in persons with HIV infection. In such patients, targeting specific symptoms may improve health-related quality of life and reduce disability. [ABSTRACT FROM AUTHOR]

Context: High-dose chemotherapy poses considerable challenges to emesis management. Although prior studies suggest that acupuncture may reduce nausea and emesis, it is unclear whether such benefit comes from the nonspecific effects of attention and clinician-patient interaction.Objective: To compare the effectiveness of electroacupuncture vs minimal needling and mock electrical stimulation or antiemetic medications alone in controlling emesis among patients undergoing a highly emetogenic chemotherapy regimen.Design: Three-arm, parallel-group, randomized controlled trial conducted from March 1996 to December 1997, with a 5-day study period and a 9-day follow-up.Setting: Oncology center at a university medical center.Patients: One hundred four women (mean age, 46 years) with high-risk breast cancer.Interventions: Patients were randomly assigned to receive low-frequency electroacupuncture at classic antiemetic acupuncture points once daily for 5 days (n = 37); minimal needling at control points with mock electrostimulation on the same schedule (n = 33); or no adjunct needling (n = 34). All patients received concurrent triple antiemetic pharmacotherapy and high-dose chemotherapy (cyclophosphamide, cisplatin, and carmustine).Main Outcome Measures: Total number of emesis episodes occurring during the 5-day study period and the proportion of emesis-free days, compared among the 3 groups.Results: The number of emesis episodes occurring during the 5 days was lower for patients receiving electroacupuncture compared with those receiving minimal needling or pharmacotherapy alone (median number of episodes, 5, 10, and 15, respectively; P<.001). The electroacupuncture group had fewer episodes of emesis than the minimal needling group (P<.001), whereas the minimal needling group had fewer episodes of emesis than the antiemetic pharmacotherapy alone group (P =.01). The differences among groups were not significant during the 9-day follow-up period (P =.18).Conclusions: In this study of patients with breast cancer receiving high-dose chemotherapy, adjunct electroacupuncture was more effective in controlling emesis than minimal needling or antiemetic pharmacotherapy alone, although the observed effect had limited duration. JAMA. 2000;284:2755-2761. [ABSTRACT FROM AUTHOR]

Background: Disability in instrumental activities of daily living (IADLs) or activities of daily living (ADLs) is an indicator of health risk. The inclusion of these items in population screens may be limited by variation in item performance across gender and age groups. Further, identification of shortened lists may encourage inclusion of these items in screens.Methods: We applied item response theory (IRT) methods to assess the responses of 9865 community-dwelling elders in the 1993 Medicare Current Beneficiary Survey to 11 IADL/ADL items. Items were classified as "receive help/not receive help" for the overall population and stratified by age and gender. We assessed the same IADL/ADL items using responses classified as "difficulty/no difficulty." After eliminating items that performed poorly, we performed all-subsets analyses to identify abbreviated sets of items that would select the highest proportion of persons with IADL/ADL disability.Results: Responses classified in receive help format showed consistency by gender and age group. Changing the response classification to difficulty/no difficulty influenced the reported order and relationship of IADL/ADL items. Receipt of help for any one of five items--shopping, doing light housework, walking, bathing, or managing finances--identified 93% of individuals receiving help with any IADL/ADL. A slightly different set of five items--walking, shopping, transferring, doing light housework, or bathing--identified 91% of persons reporting difficulty with any IADL or ADL.Conclusions: The relationship of IADL and ADL items to the underlying construct of disability was similar for men and women. The relationship was also similar for oldest-old and younger-old individuals. This study also identified abbreviated lists of disability items that can be used to efficiently screen community-dwelling elders for the presence of IADL/ADL disability. [ABSTRACT FROM AUTHOR]