Search

Reliability refers to how measurements can produce consistent results and are crucial for any scientific research measurement. Intraclass correlation coefficient (ICC) is the most widely used method to determine the reproducibility of measurements of various statistical techniques. Calculated ICC and its confidence interval that reveal the underlying sampling distribution may help detect an experimental method's ability to identify systematic differences between research participants in a test. This study aimed to introduce a new SAS macro, ICC6, for calculating different ICC forms and their confidence intervals. A SAS macro that employs the PROC GLM procedure in SAS was created to generate two-way random effects (ANOVA) estimates. A simulated dataset was used to input the macro to calculate the point estimates for different ICCs. The ICC forms' upper and lower confidence interval limits were calculated using the F statistics distribution. Our SAS macro provides a complete set of various ICC forms and their confidence intervals. A validation analysis using commercial software packages STATA and SPSS delivered identical results. A development of SAS methodology using publicly available statistical approaches in estimating six distinct forms of ICC and their confidence intervals has been reported in this article. This work is an extension of general methodology supported by a few other statistical software packages to SAS. [ABSTRACT FROM AUTHOR]

The quality of propensity scores is traditionally measured by assessing how well they make the distributions of covariates in the treatment and control groups match, which we refer to as 'good balance'. Good balance guarantees less biased estimates of the treatment effect. However, the cost of achieving good balance is that the variance of the estimates increases due to a reduction in effective sample size, either through the introduction of propensity score weights or dropping cases when propensity score matching. In this paper, we investigate whether it is best to optimize the balance or to settle for a less than optimal balance and use double robust estimation to adjust for remaining differences. We compare treatment effect estimates from regression, propensity score weighting, and double robust estimation with varying levels of effort expended to achieve balance using data from a study about the differences in outcomes by HIV status in heterosexually active homeless men residing in Los Angeles. Because of how costly data collection efforts are for this population, it is important to find an alternative estimation method that does not reduce effective sample size as much as methods that aggressively aim to optimize balance. Results from a simulation study suggest that there are instances in which we can obtain more precise treatment effect estimates without increasing bias too much by using a combination of regression and propensity score weights that achieve a less than optimal balance. There is a bias-variance tradeoff at work in propensity score estimation; every step toward better balance usually means an increase in variance and at some point a marginal decrease in bias may not be worth the associated increase in variance. [ABSTRACT FROM AUTHOR]

The marginalized two-part models, including the marginalized zero-inflated Poisson and negative binomial models, have been proposed in the literature for modelling cross-sectional healthcare utilization count data with excess zeroes and overdispersion. The motivation for these proposals was to directly capture the overall marginal effects and to avoid post-modelling effect calculations that are needed for the non-marginalized conventional two-part models. However, are marginalized two-part models superior to non-marginalized two-part models because of their structural property? Is it true that the marginalized two-part models can provide direct marginal inference? This article aims to answer these questions through a comprehensive investigation. We first summarize the existing non-marginalized and marginalized two-part models and then develop marginalized hurdle Poisson and negative binomial models for cross-sectional count data with abundant zero counts. Our interest in the investigation lies particularly in the (average) marginal effect and (average) incremental effect and the comparison of these effects. The estimators of these effects are presented, and variance estimators are derived by using delta methods and Taylor series approximations. Though the marginalized models attract attention because of the alleged convenience of direct marginal inference, we provide evidence for the impact of model misspecification of the marginalized models over the conventional models, and provide evidence for the importance of goodness-of-fit evaluation and model selection in differentiating between the marginalized and non-marginalized models. An empirical analysis of the German Socioeconomic Panel data is presented. [ABSTRACT FROM AUTHOR]

Provider profiling as a means to describe and compare the performance of health care professionals has gained momentum in the past decade. As a key component of pay-for-performance programs profiling has been increasingly used to identify top-performing providers. However, rigorous examination of the performance of statistical methods for profiling when used to classify top-performing providers is lacking. The objective of this study was to compare the classification accuracy of three methods for identifying providers exceeding performance thresholds and to analyze data on satisfaction with mental health care providers at Group Health Cooperative using these methods. Questionnaire data on patient satisfaction with mental health care providers at Group Health Cooperative was collected between April 2008 and January 2010. Simulated data were used to compare the classification accuracy of alternative statistical methods. We evaluated sensitivity, specificity, and root mean squared error of alternative statistical methods using simulated data. For Group Health providers, we compared agreement of alternative approaches to classification. We found that when between-provider variability in performance was low, all three methods exhibited poor classification accuracy. When used to evaluate mental health care provider performance, we found substantial uncertainty in the estimates and poor agreement across methods. Based on these findings, we recommend providing uncertainty estimates for provider rankings and caution against the use of any classification method when between-provider variability is low. [ABSTRACT FROM AUTHOR]

In order to create an empirically derived parsimonious typology of physician financial incentives that will be useful for future research, we used data from the nationally representative 2004–2005 Community Tracking Study Physician Survey (N = 6,628). Linear regression analyses informed by economic theory were used to identify the combinations of incentives associated with an overall financial incentive to expand services to individual patients. The approach was validated using two nonparametric methods (CART analysis and data mining techniques) and by examining the relationship between the resulting typology and other measures of physician behavior including hours worked, visit volume, and specialty-adjusted income. Of the 6,628 physicians surveyed, approximately 25% (1,605) reported an overall incentive to increase services and 75% (5,023) reported either neutral incentives or incentives to decrease services. Men, who were approximately 75% of respondents, were slightly more likely to report incentives to increase services ( P < 0.05). There were no differences in reported incentives according to specialty. We created two typologies (one with eleven categories and the other with a collapsed set of six categories) based on combinations of variables measuring ownership, base compensation methods, and financial incentives. The percentage with an overall incentive to increase services ranges from 6% for employed physicians compensated via fixed salary to 36.7% for owners in low capitation environments with either individual or practice level productivity incentives. The criterion validity of the typology was established by examining the relationship with adjusted physician income, hours worked, and visit volume, which showed generally consistent relationships in the expected direction. A parsimonious typology consisting of six mutually exclusive groups reasonably captures the continuum of incentives to increase service delivery experienced by physicians. [ABSTRACT FROM AUTHOR]

Standard approaches for determining weights when calculating a composite measure of health care quality from individual quality indicators (QIs) include equal weighting, opportunity-based weights, and judgment-based weights. Benefit-of-the-doubt approaches have not been used in the health services area, though one has been used to calculate composite measures for profiling countries. Underlying these approaches is the assumption that relative performance on a set of indicators is, at least to some extent, a revealed preference by the organizational unit about the relative importance of the indicators. A benefit-of-the-doubt approach recognizes these revealed preferences by assigning higher weights to indicators on which performance is better and lower weights to indicators on which performance is poorer. We consider two benefit-of-the-doubt approaches. The first uses simple linear programming (LP) models; the second uses data envelopment analysis (DEA), the way in which the benefit-of-the-doubt approach has been previously implemented. In both cases, constraints are added to limit weight adjustments to some percentage of policy-determined baseline weights. Using both standard and benefit-of-the-doubt approaches, composite scores are calculated from data on five QIs from 32 Department of Veterans Affairs (VA) nursing homes. We examine the tradeoff between the level of allowable weight adjustment and impact on facility rankings. If weights are constrained to be within 75% of baseline weights, all approaches identify pretty much the same high performing facilities. Weights from benefit-of-the-doubt approaches, because they are able to reflect local preferences and conditions, should be attractive to facilities and, in a collaborative environment, to policy makers. [ABSTRACT FROM AUTHOR]