Reliability of the American Community Survey Estimates of Risk-Adjusted Readmission Rankings for Hospitals Before and After Peer Group Stratification

Key Points Question Is the quality of American Community Survey estimates associated with changes in hospital readmission rankings after socioeconomic risk adjustment? Findings In this cross-sectional study of 96 278 hospital admissions for acute myocardial infarction, pneumonia, and congestive heart failure included in the 2014 New York State Health Cost and Utilization Project, compared with previous stratification by safety-net hospital designation, the new peer group–based stratification system was associated with improved reliability of American Community Survey socioeconomic status estimates. Meaning Poor reliability in the American Community Survey can conceal and distort readmission rates for hospitals, and the use of peer group–based performance measurement is associated with a reduction, but not elimination, of the impact of measurement error on risk-adjusted rates.
This cross-sectional study uses data from the New York State Health Cost and Utilization Project State Inpatient Database to examine whether the reliability of American Community Survey socioeconomic status estimates improves with the use of new peer group–based strata compared with previous stratification by safety-net hospital designation.
Importance Since the transition to the American Community Survey, data uncertainty has complicated its use for policy making and research, despite the ongoing need to identify disparities in health care outcomes. The US Centers for Medicare & Medicaid Services’ new, stratified payment adjustment method for its Hospital Readmissions Reduction Program may be able to reduce the reliance on data linkages to socioeconomic survey estimates. Objective To determine whether there are differences in the reliability of socioeconomically risk-adjusted hospital readmission rates among hospitals that serve a disproportionate share of low-income populations after stratifying hospitals into peer group–based classification groups. Design, Setting, and Participants This cross-sectional study uses data from the 2014 New York State Health Cost and Utilization Project State Inpatient Database for 96 278 hospital admissions for acute myocardial infarction, pneumonia, and congestive heart failure. The analysis included patients aged 18 years and older who were not transferred to another hospital, who were discharged alive, who did not leave the hospital against medical advice, and who were discharged before December 2014. Main Outcomes and Measures The main outcomes were 30-day hospital readmissions after acute myocardial infarction, pneumonia, and congestive heart failure assessed using hierarchical logistic regression. Results The mean (SD) age of the patients was 69.6 (16.0) years for the safety-net hospitals and 74.9 (14.7) years for the non–safety-net hospitals; 9382 (48.8%) and 7003 (48.5%) patients, respectively, were female. For safety net designations, 20% (3 of 15) of all evaluations concealed and distorted differences in risk, with factors such as poverty failing to identify similar risk of acute myocardial infarction readmission until unreliable estimates were excluded from the analysis (OR, 1.23 [95% CI, 1.00-1.52], P = .02; vs OR, 1.17 [95% CI, 0.94-1.46], P = .15). By comparison, 2 of the 60 models (3%) for the peer group–based classification altered the association between socioeconomic status and readmission risk, concealing similarities in congestive heart failure readmission when adjusted using high school completion rates (OR, 1.27 [95% CI 1.02-1.58], P = .04; vs OR, 1.23 [95% CI, 0.98-1.53], P = .06) and distorting similarities in pneumonia readmissions when accounting for the proportion of lone-parent families (OR, 1.27 [95% CI, 0.98-1.66], P = .07; vs OR, 1.35 [95% CI, 1.02-1.80], P = .04) between the lowest and highest socioeconomic status hospitals in quartile 1. Conclusions and Relevance There was greater precision in socioeconomic adjusted readmission estimates when hospitals were stratified into the new payment adjustment criteria compared with safety net designations. A contributing factor for improved reliability of American Community Survey estimates under the new payment criteria was the merging of patients from low-income neighborhoods with greater homogeneity in survey estimates into groupings similar to those for higher-income patients, whose neighborhoods often exhibit greater estimate variability. Additional efforts are needed to explore the effect of measurement error on American Community Survey–adjusted readmissions using the new peer group–based classification methods.