183 results on '"Keele, Luke"'
Search Results
2. Pre-analysis Plan for a Comparison of Matching and Black Box-based Covariate Adjustment
- Author
-
Keele, Luke and Small, Dylan S.
- Published
- 2021
- Full Text
- View/download PDF
3. Matching with attention to effect modification in a data challenge
- Author
-
Keele, Luke and Pimentel, Samuel D.
- Published
- 2021
- Full Text
- View/download PDF
4. Book Review of Explanation in Causal Inference: Methods of Mediation and Interaction (author: T.J. Vanderweele)
- Author
-
Keele, Luke J.
- Published
- 2021
- Full Text
- View/download PDF
5. Nonparametric identification and efficient estimation of causal effects with instrumental variables
- Author
-
Levis, Alexander W., Kennedy, Edward H., and Keele, Luke
- Subjects
Statistics - Methodology - Abstract
Instrumental variables are widely used in econometrics and epidemiology for identifying and estimating causal effects when an exposure of interest is confounded by unmeasured factors. Despite this popularity, the assumptions invoked to justify the use of instruments differ substantially across the literature. Similarly, statistical approaches for estimating the resulting causal quantities vary considerably, and often rely on strong parametric assumptions. In this work, we compile and organize structural conditions that nonparametrically identify conditional average treatment effects, average treatment effects among the treated, and local average treatment effects, with a focus on identification formulae invoking the conditional Wald estimand. Moreover, we build upon existing work and propose nonparametric efficient estimators of functionals corresponding to marginal and conditional causal contrasts resulting from the various identification paradigms. We illustrate the proposed methods on an observational study examining the effects of operative care on adverse events for cholecystitis patients, and a randomized trial assessing the effects of market participation on political views., Comment: 46 pages, 2 figures
- Published
- 2024
6. Flexibly Estimating and Interpreting Heterogeneous Treatment Effects of Laparoscopic Surgery for Cholecystitis Patients
- Author
-
Bonvini, Matteo, Zeng, Zhenghao, Yu, Miaoqing, Kennedy, Edward H., and Keele, Luke
- Subjects
Statistics - Methodology - Abstract
Laparoscopic surgery has been shown through a number of randomized trials to be an effective form of treatment for cholecystitis. Given this evidence, one natural question for clinical practice is: does the effectiveness of laparoscopic surgery vary among patients? It might be the case that, while the overall effect is positive, some patients treated with laparoscopic surgery may respond positively to the intervention while others do not or may be harmed. In our study, we focus on conditional average treatment effects to understand whether treatment effects vary systematically with patient characteristics. Recent methodological work has developed a meta-learner framework for flexible estimation of conditional causal effects. In this framework, nonparametric estimation methods can be used to avoid bias from model misspecification while preserving statistical efficiency. In addition, researchers can flexibly and effectively explore whether treatment effects vary with a large number of possible effect modifiers. However, these methods have certain limitations. For example, conducting inference can be challenging if black-box models are used. Further, interpreting and visualizing the effect estimates can be difficult when there are multi-valued effect modifiers. In this paper, we develop new methods that allow for interpretable results and inference from the meta-learner framework for heterogeneous treatment effects estimation. We also demonstrate methods that allow for an exploratory analysis to identify possible effect modifiers. We apply our methods to a large database for the use of laparoscopic surgery in treating cholecystitis. We also conduct a series of simulation studies to understand the relative performance of the methods we develop. Our study provides key guidelines for the interpretation of conditional causal effects from the meta-learner framework., Comment: 48 pages, 7 figures
- Published
- 2023
7. Doubly robust machine learning for an instrumental variable study of surgical care for cholecystitis
- Author
-
Takatsu, Kenta, Levis, Alexander W., Kennedy, Edward, Kelz, Rachel, and Keele, Luke
- Subjects
Statistics - Applications ,Statistics - Methodology - Abstract
Comparative effectiveness research frequently employs the instrumental variable design since randomized trials can be infeasible for many reasons. In this study, we investigate and compare treatments for emergency cholecystitis -- inflammation of the gallbladder. A standard treatment for cholecystitis is surgical removal of the gallbladder, while alternative non-surgical treatments include managed care and pharmaceutical options. As randomized trials are judged to violate the principle of equipoise, we consider an instrument for operative care: the surgeon's tendency to operate. Standard instrumental variable estimation methods, however, often rely on parametric models that are prone to bias from model misspecification. We outline instrumental variable estimation methods based on the doubly robust machine learning framework. These methods enable us to employ various machine learning techniques for nuisance parameter estimation and deliver consistent estimates and fast rates of convergence for valid inference. We use these methods to estimate the primary target causal estimand in an IV design. Additionally, we expand these methods to develop estimators for heterogeneous causal effects, profiling principal strata, and a sensitivity analyses for a key instrumental variable assumption. We conduct a simulation study to demonstrate scenarios where more flexible estimation methods outperform standard methods. Our findings indicate that operative care is generally more effective for cholecystitis patients, although the benefits of surgery can be less pronounced for key patient subgroups.
- Published
- 2023
8. Minimax optimal subgroup identification
- Author
-
Bonvini, Matteo, Kennedy, Edward H., and Keele, Luke J.
- Subjects
Statistics - Methodology - Abstract
Quantifying treatment effect heterogeneity is a crucial task in many areas of causal inference, e.g. optimal treatment allocation and estimation of subgroup effects. We study the problem of estimating the level sets of the conditional average treatment effect (CATE), identified under the no-unmeasured-confounders assumption. Given a user-specified threshold, the goal is to estimate the set of all units for whom the treatment effect exceeds that threshold. For example, if the cutoff is zero, the estimand is the set of all units who would benefit from receiving treatment. Assigning treatment just to this set represents the optimal treatment rule that maximises the mean population outcome. Similarly, cutoffs greater than zero represent optimal rules under resource constraints. The level set estimator that we study follows the plug-in principle and consists of simply thresholding a good estimator of the CATE. While many CATE estimators have been recently proposed and analysed, how their properties relate to those of the corresponding level set estimators remains unclear. Our first goal is thus to fill this gap by deriving the asymptotic properties of level set estimators depending on which estimator of the CATE is used. Next, we identify a minimax optimal estimator in a model where the CATE, the propensity score and the outcome model are Holder-smooth of varying orders. We consider data generating processes that satisfy a margin condition governing the probability of observing units for whom the CATE is close to the threshold. We investigate the performance of the estimators in simulations and illustrate our methods on a dataset used to study the effects on mortality of laparoscopic vs open surgery in the treatment of various conditions of the colon., Comment: 38 pages, 4 figures
- Published
- 2023
9. Improving instrumental variable estimators with post-stratification
- Author
-
Pashley, Nicole E., Keele, Luke, and Miratrix, Luke W.
- Subjects
Statistics - Methodology - Abstract
Experiments studying get-out-the-vote (GOTV) efforts estimate the causal effect of various mobilization efforts on voter turnout. However, there is often substantial noncompliance in these studies. A usual approach is to use an instrumental variable (IV) analysis to estimate impacts for compliers, here being those actually contacted by the investigators. Unfortunately, popular IV estimators can be unstable in studies with a small fraction of compliers. We explore post-stratifying the data (e.g., taking a weighted average of IV estimates within each stratum) using variables that predict complier status (and, potentially, the outcome) to mitigate this. We present the benefits of post-stratification in terms of bias, variance, and improved standard error estimates, and provide a finite-sample asymptotic variance formula. We also compare the performance of different IV approaches and discuss the advantages of our design-based post-stratification approach over incorporating compliance-predictive covariates into the two-stage least squares estimator. In the end, we show that covariates predictive of compliance can increase precision, but only if one is willing to make a bias-variance trade-off by down-weighting or dropping strata with few compliers. By contrast, standard approaches such as two-stage least squares fail to use such information. We finally examine the benefits of our approach in two GOTV applications.
- Published
- 2023
10. A Guide to Regression Discontinuity Designs in Medical Applications
- Author
-
Cattaneo, Matias D., Keele, Luke, and Titiunik, Rocio
- Subjects
Statistics - Methodology ,Economics - Econometrics ,Statistics - Applications - Abstract
We present a practical guide for the analysis of regression discontinuity (RD) designs in biomedical contexts. We begin by introducing key concepts, assumptions, and estimands within both the continuity-based framework and the local randomization framework. We then discuss modern estimation and inference methods within both frameworks, including approaches for bandwidth or local neighborhood selection, optimal treatment effect point estimation, and robust bias-corrected inference methods for uncertainty quantification. We also overview empirical falsification tests that can be used to support key assumptions. Our discussion focuses on two particular features that are relevant in biomedical research: (i) fuzzy RD designs, which often arise when therapeutic treatments are based on clinical guidelines but patients with scores near the cutoff are treated contrary to the assignment rule; and (ii) RD designs with discrete scores, which are ubiquitous in biomedical applications. We illustrate our discussion with three empirical applications: the effect of CD4 guidelines for anti-retroviral therapy on retention of HIV patients in South Africa, the effect of genetic guidelines for chemotherapy on breast cancer recurrence in the United States, and the effects of age-based patient cost-sharing on healthcare utilization in Taiwan. We provide replication materials employing publicly available statistical software in Python, R and Stata, offering researchers all necessary tools to conduct an RD analysis.
- Published
- 2023
11. Covariate-assisted bounds on causal effects with instrumental variables
- Author
-
Levis, Alexander W., Bonvini, Matteo, Zeng, Zhenghao, Keele, Luke, and Kennedy, Edward H.
- Subjects
Statistics - Methodology - Abstract
When an exposure of interest is confounded by unmeasured factors, an instrumental variable (IV) can be used to identify and estimate certain causal contrasts. Identification of the marginal average treatment effect (ATE) from IVs relies on strong untestable structural assumptions. When one is unwilling to assert such structure, IVs can nonetheless be used to construct bounds on the ATE. Famously, Balke and Pearl (1997) proved tight bounds on the ATE for a binary outcome, in a randomized trial with noncompliance and no covariate information. We demonstrate how these bounds remain useful in observational settings with baseline confounders of the IV, as well as randomized trials with measured baseline covariates. The resulting bounds on the ATE are non-smooth functionals, and thus standard nonparametric efficiency theory is not immediately applicable. To remedy this, we propose (1) under a novel margin condition, influence function-based estimators of the bounds that can attain parametric convergence rates when the nuisance functions are modeled flexibly, and (2) estimators of smooth approximations of these bounds. We propose extensions to continuous outcomes, explore finite sample properties in simulations, and illustrate the proposed estimators in an observational study targeting the effect of higher education on wages., Comment: 42 pages, 2 figures
- Published
- 2023
12. Approximate Balancing Weights for Clustered Observational Study Designs
- Author
-
Keele, Luke, Ben-Michael, Eli, and Page, Lindsay
- Subjects
Statistics - Methodology - Abstract
In a clustered observational study, a treatment is assigned to groups and all units within the group are exposed to the treatment. We develop a new method for statistical adjustment in clustered observational studies using approximate balancing weights, a generalization of inverse propensity score weights that solve a convex optimization problem to find a set of weights that directly minimize a measure of covariate imbalance, subject to an additional penalty on the variance of the weights. We tailor the approximate balancing weights optimization problem to both adjustment sets by deriving an upper bound on the mean square error for each case and finding weights that minimize this upper bound, linking the level of covariate balance to a bound on the bias. We implement the procedure by specializing the bound to a random cluster-level effects model, leading to a variance penalty that incorporates the signal signal-to-noise ratio and penalizes the weight on individuals and the total weight on groups differently according to the the intra-class correlation.
- Published
- 2023
13. Using Balancing Weights to Target the Treatment Effect on the Treated when Overlap is Poor
- Author
-
Ben-Michael, Eli and Keele, Luke
- Subjects
Statistics - Methodology ,Statistics - Applications - Abstract
Inverse probability weights are commonly used in epidemiology to estimate causal effects in observational studies. Researchers can typically focus on either the average treatment effect or the average treatment effect on the treated with inverse probability weighting estimators. However, when overlap between the treated and control groups is poor, this can produce extreme weights that can result in biased estimates and large variances. One alternative to inverse probability weights are overlap weights, which target the population with the most overlap on observed characteristics. While estimates based on overlap weights produce less bias in such contexts, the causal estimand can be difficult to interpret. One alternative to inverse probability weights are balancing weights, which directly target imbalances during the estimation process. Here, we explore whether balancing weights allow analysts to target the average treatment effect on the treated in cases where inverse probability weights are biased due to poor overlap. We conduct three simulation studies and an empirical application. We find that in many cases, balancing weights allow the analyst to still target the average treatment effect on the treated even when overlap is poor. We show that while overlap weights remain a key tool for estimating causal effects, more familiar estimands can be targeted by using balancing weights instead of inverse probability weights.
- Published
- 2022
14. Estimating Racial Disparities in Emergency General Surgery
- Author
-
Ben-Michael, Eli, Feller, Avi, Kelz, Rachel, and Keele, Luke
- Subjects
Statistics - Applications ,Statistics - Methodology - Abstract
Research documents that Black patients experience worse general surgery outcomes than white patients in the United States. In this paper, we focus on an important but less-examined category: the surgical treatment of emergency general surgery (EGS) conditions, which refers to medical emergencies where the injury is "endogenous," such as a burst appendix. Our goal is to assess racial disparities for common outcomes after EGS treatment using an administrative database of hospital claims in New York, Florida, and Pennsylvania, and to understand the extent to which differences are attributable to patient-level risk factors versus hospital-level factors. To do so, we use a class of linear weighting estimators that re-weight white patients to have a similar distribution of baseline characteristics as Black patients. This framework nests many common approaches, including matching and linear regression, but offers important advantages over these methods in terms of controlling imbalance between groups, minimizing extrapolation, and reducing computation time. Applying this approach to the claims data, we find that disparities estimates that adjust for the admitting hospital are substantially smaller than estimates that adjust for patient baseline characteristics only, suggesting that hospital-specific factors are important drivers of racial disparities in EGS outcomes.
- Published
- 2022
15. Nonparametric identification of causal effects in clustered observational studies with differential selection
- Author
-
Ye, Ting, Westling, Ted, Page, Lindsay, and Keele, Luke
- Subjects
Statistics - Methodology ,Statistics - Applications - Abstract
The clustered observational study (COS) design is the observational study counterpart to the clustered randomized trial. In a COS, a treatment is assigned to intact groups, and all units within the group are exposed to the treatment. However, the treatment is non-randomly assigned. COSs are common in both education and health services research. In education, treatments may be given to all students within some schools but withheld from all students in other schools. In health studies, treatments may be applied to clusters such as hospitals or groups of patients treated by the same physician. In this manuscript, we study the identification of causal effects in clustered observational study designs. We focus on the prospect of differential selection of units to clusters, which occurs when the units' cluster selections depend on the clusters' treatment assignments. Extant work on COSs has made an implicit assumption that rules out the presence of differential selection. We derive the identification results for designs with differential selection and that contexts with differential cluster selection require different adjustment sets than standard designs. We outline estimators for designs with and without differential selection. Using a series of simulations, we outline the magnitude of the bias that can occur with differential selection. We then present two empirical applications focusing on the likelihood of differential selection.
- Published
- 2022
16. Covariate Adjustment in Regression Discontinuity Designs
- Author
-
Cattaneo, Matias D., Keele, Luke, and Titiunik, Rocio
- Subjects
Statistics - Methodology ,Economics - Econometrics - Abstract
The Regression Discontinuity (RD) design is a widely used non-experimental method for causal inference and program evaluation. While its canonical formulation only requires a score and an outcome variable, it is common in empirical work to encounter RD analyses where additional variables are used for adjustment. This practice has led to misconceptions about the role of covariate adjustment in RD analysis, from both methodological and empirical perspectives. In this chapter, we review the different roles of covariate adjustment in RD designs, and offer methodological guidance for its correct use.
- Published
- 2021
17. The Design of Clustered Observational Studies in Education
- Author
-
Page, Lindsay C., Lenard, Matthew A., and Keele, Luke
- Abstract
Clustered observational studies (COSs) are a critical analytic tool for educational effectiveness research. We present a design framework for the development and critique of COSs. The framework is built on the counterfactual model for causal inference and promotes the concept of designing COSs that emulate the targeted randomized trial that would have been conducted were it feasible. We emphasize the key role of understanding the assignment mechanism to study design. We review methods for statistical adjustment and highlight a recently developed form of matching designed specifically for COSs. We review how regression models can be profitably combined with matching and note best practices for estimates of statistical uncertainty. Finally, we review how sensitivity analyses can determine whether conclusions are sensitive to bias from potential unobserved confounders. We demonstrate concepts with an evaluation of a summer school reading intervention in a large U.S. school district.
- Published
- 2020
18. Profiling Compliers in Instrumental Variables Designs
- Author
-
Hangartner, Dominik, Marbach, Moritz, Henckel, Leonard, Maathuis, Marloes H., Kelz, Rachel R., and Keele, Luke
- Subjects
Statistics - Methodology - Abstract
Instrumental variable (IV) analyses are becoming common in health services research and epidemiology. IV analyses can be used both to analyze randomized trials with noncompliance and as a form of natural experiment. In these analyses, investigators often adopt a monotonicity assumption, which implies that the relevant effect only applies to a subset of the study population known as compliers. Since the estimated effect is not the average treatment effect of the study population, it is important to compare the characteristics of compliers and non-compliers. Profiling compliers and non-compliers is necessary to understand what subpopulation the researcher is making inferences about, and an important first step in evaluating the external validity (or lack thereof) of the IV estimate for compliers. Here, we discuss the assumptions necessary for profiling, which are weaker than the assumptions necessary for identifying the local average treatment effect if the instrument is randomly assigned. We then outline a simple and general method to characterize compliers and noncompliers using baseline covariates. Next, we extend current methods by deriving standard errors for these estimates. We demonstrate these methods using an IV known as tendency to operate (TTO) from health services research.
- Published
- 2021
19. A note on post-treatment selection in studying racial discrimination in policing
- Author
-
Zhao, Qingyuan, Keele, Luke J, Small, Dylan S, and Joffe, Marshall M
- Subjects
Statistics - Applications ,Statistics - Methodology ,62D20 (Primary) 62P25 (Secondary) - Abstract
We discuss some causal estimands used to study racial discrimination in policing. A central challenge is that not all police-civilian encounters are recorded in administrative datasets and available to researchers. One possible solution is to consider the average causal effect of race conditional on the civilian already being detained by the police. We find that such an estimand can be quite different from the more familiar ones in causal inference and needs to be interpreted with caution. We propose using an estimand new for this context -- the causal risk ratio, which has more transparent interpretation and requires weaker identification assumptions. We demonstrate this through a reanalysis of the NYPD Stop-and-Frisk dataset. Our reanalysis shows that the naive estimator that ignores the post-treatment selection in administrative records may severely underestimate the disparity in police violence between minorities and whites in these and similar data., Comment: Accepted for publication in the American Political Science Review on 14th June, 2021
- Published
- 2020
20. Hospital Quality Risk Standardization via Approximate Balancing Weights
- Author
-
Keele, Luke, Ben-Michael, Eli, Feller, Avi, Kelz, Rachel, and Miratrix, Luke
- Subjects
Statistics - Applications - Abstract
Comparing outcomes across hospitals, often to identify underperforming hospitals, is a critical task in health services research. However, naive comparisons of average outcomes, such as surgery complication rates, can be misleading because hospital case mixes differ -- a hospital's overall complication rate may be lower due to more effective treatments or simply because the hospital serves a healthier population overall. In this paper, we develop a method of ``direct standardization'' where we re-weight each hospital patient population to be representative of the overall population and then compare the weighted averages across hospitals. Adapting methods from survey sampling and causal inference, we find weights that directly control for imbalance between the hospital patient mix and the target population, even across many patient attributes. Critically, these balancing weights can also be tuned to preserve sample size for more precise estimates. We also derive principled measures of statistical precision, and use outcome modeling and Bayesian shrinkage to increase precision and account for variation in hospital size. We demonstrate these methods using claims data from Pennsylvania, Florida, and New York, estimating standardized hospital complication rates for general surgery patients. We conclude with a discussion of how to detect low performing hospitals.
- Published
- 2020
21. A Negative Correlation Strategy for Bracketing in Difference-in-Differences
- Author
-
Ye, Ting, Keele, Luke, Hasegawa, Raiden, and Small, Dylan S.
- Subjects
Statistics - Methodology ,Economics - Econometrics ,Statistics - Applications - Abstract
The method of difference-in-differences (DID) is widely used to study the causal effect of policy interventions in observational studies. DID employs a before and after comparison of the treated and control units to remove bias due to time-invariant unmeasured confounders under the parallel trends assumption. Estimates from DID, however, will be biased if the outcomes for the treated and control units evolve differently in the absence of treatment, namely if the parallel trends assumption is violated. We propose a general identification strategy that leverages two groups of control units whose outcomes relative to the treated units exhibit a negative correlation, and achieves partial identification of the average treatment effect for the treated. The identified set is of a union bounds form that involves the minimum and maximum operators, which makes the canonical bootstrap generally inconsistent and naive methods overly conservative. By utilizing the directional inconsistency of the bootstrap distribution, we develop a novel bootstrap method to construct uniformly valid confidence intervals for the identified set and parameter of interest when the identified set is of a union bounds form, and we establish the method's theoretical properties. We develop a simple falsification test and sensitivity analysis. We apply the proposed strategy for bracketing to study whether minimum wage laws affect employment levels.
- Published
- 2020
22. Protocol for a Study of the Effect of Surface Mining in Central Appalachia on Adverse Birth Outcomes
- Author
-
Small, Dylan S., Firth, Dan, Keele, Luke, Huber, Matthew, Passarella, Molly, Lorch, Scott, and Burris, Heather
- Subjects
Statistics - Applications - Abstract
Surface mining has become a major method of coal mining in Central Appalachia alongside the traditional underground mining. Concerns have been raised about the health effects of this surface mining, particularly mountaintop removal mining where coal is mined upon steep mountaintops by removing the mountaintop through clearcutting forests and explosives. We have designed a matched observational study to assess the effects of surface mining in Central Appalachia on adverse birth outcomes. This protocol describes for the study the background and motivation, the sample selection and the analysis plan.
- Published
- 2020
23. Comparing the Performance of Statistical Adjustment Methods By Recovering the Experimental Benchmark from the REFLUX Trial
- Author
-
Keele, Luke, O'Neill, Stephen, and Grieve, Richard
- Subjects
Statistics - Applications - Abstract
Much evidence in comparative effectiveness research is based on observational studies. Researchers who conduct observational studies typically assume that there are no unobservable differences between the treated and control groups. Treatment effects are estimated after adjusting for observed differences between treated and controls. However, treatment effect estimates may be biased due to model misspecification. That is, if the method of treatment effect estimation imposes unduly strong functional form assumptions, treatment effect estimates may be significantly biased. In this study, we compare the performance of a wide variety of treatment effect estimation methods. We do so within the context of the REFLUX study from the UK. In REFLUX, after study qualification, participants were enrolled in either a randomized trial arm or patient preference arm. In the randomized trial, patients were randomly assigned to either surgery or medical management. In the patient preference arm, participants selected to either have surgery or medical management. We attempt to recover the treatment effect estimate from the randomized trial arm using the data from the patient preference arm of the study. We vary the method of treatment effect estimation and record which methods are successful and which are not. We apply over 20 different methods including standard regression models as well as advanced machine learning methods. We find that simple propensity score matching methods perform the worst. We also find significant variation in performance across methods. The wide variation in performance suggests analysts should use multiple methods of estimation as a robustness check.
- Published
- 2020
24. Impact of Medicaid expansion on outcomes after abdominal aortic aneurysm repair
- Author
-
Ramadan, Omar I., Kelz, Rachel R., Sharpe, James E., Wirtalla, Christopher J., Keele, Luke J., Harhay, Michael O., Roberts, Sanford E., and Wang, Grace J.
- Published
- 2023
- Full Text
- View/download PDF
25. Stronger instruments and refined covariate balance in an observational study of the effectiveness of prompt admission to intensive care units
- Author
-
Keele, Luke, Harris, Steve, Pimentel, Samuel D, and Grieve, Richard
- Subjects
Bioengineering ,Generic health relevance ,Statistics ,Econometrics ,Demography ,Statistics & Probability - Abstract
Instrumental variable methods, subject to appropriate identification assumptions, enable consistent estimation of causal effects in the presence of unobserved confounding. Near–far matching has been proposed as one analytic method to improve inference by strengthening the effect of the instrument on the exposure and balancing observable characteristics between groups of subjects with low and high values of the instrument. However, in settings with hierarchical data (e.g. patients nested within hospitals), or where several covariate interactions must be balanced, conventional near–far matching algorithms may fail to achieve the requisite covariate balance. We develop a new matching algorithm, that combines near–far matching with refined covariate balance, to balance large numbers of nominal covariates while also strengthening the instrumental variable. This extension of near–far matching is motivated by a case-study that aims to identify the causal effect of prompt admission to an intensive care unit on 7-day and 28-day mortality.
- Published
- 2020
26. Biased Encouragements and Heterogeneous Effects in an Instrumental Variable Study of Emergency General Surgical Outcomes
- Author
-
Fogarty, Colin B., Lee, Kwonsang, Kelz, Rachel R., and Keele, Luke J.
- Subjects
Statistics - Applications - Abstract
We investigate the efficacy of surgical versus non-surgical management for two gastrointestinal conditions, colitis and diverticulitis, using observational data. We deploy an instrumental variable design with surgeons' tendencies to operate as an instrument. Assuming instrument validity, we find that non-surgical alternatives can reduce both hospital length of stay and the risk of complications, with estimated effects larger for septic patients than for non-septic patients. The validity of our instrument is plausible but not ironclad, necessitating a sensitivity analysis. Existing sensitivity analyses for IV designs assume effect homogeneity, unlikely to hold here because of patient-specific physiology. We develop a new sensitivity analysis that accommodates arbitrary effect heterogeneity and exploits components explainable by observed features. We find that the results for non-septic patients prove more robust to hidden bias despite having smaller estimated effects. For non-septic patients, two individuals with identical observed characteristics would have to differ in their odds of assignment to a high tendency to operate surgeon by a factor of 2.34 to overturn our finding of a benefit for non-surgical management in reducing length of stay. For septic patients, this value is only 1.64. Simulations illustrate that this phenomenon may be explained by differences in within-group heterogeneity.
- Published
- 2019
27. Evaluating A Key Instrumental Variable Assumption Using Randomization Tests
- Author
-
Branson, Zach and Keele, Luke
- Subjects
Statistics - Methodology - Abstract
Instrumental variable (IV) analyses are becoming common in health services research and epidemiology. Most IV analyses use naturally occurring instruments, such as distance to a hospital. In these analyses, investigators must assume the instrument is as-if randomly assigned. This assumption cannot be tested directly, but it can be falsified. Most falsification tests in the literature compare relative prevalence or bias in observed covariates between the instrument and the exposure. These tests require investigators to make a covariate-by-covariate judgment about the validity of the IV design. Often, only some of the covariates are well-balanced, making it unclear if as-if randomization can be assumed for the instrument across all covariates. We propose an alternative falsification test that compares IV balance or bias to the balance or bias that would have been produced under randomization. A key advantage of our test is that it allows for global balance measures as well as easily interpretable graphical comparisons. Furthermore, our test does not rely on any parametric assumptions and can be used to validly assess if the instrument is significantly closer to being as-if randomized than the exposure. We demonstrate our approach on a recent IV application that uses bed availability in the intensive care unit (ICU) as an instrument for admission to the ICU., Comment: 19 pages
- Published
- 2019
28. Patterns of Effects and Sensitivity Analysis for Differences-in-Differences
- Author
-
Keele, Luke J., Small, Dylan S., Hsu, Jesse Y., and Fogarty, Colin B.
- Subjects
Statistics - Applications - Abstract
Applied analysts often use the differences-in-differences (DID) method to estimate the causal effect of policy interventions with observational data. The method is widely used, as the required before and after comparison of a treated and control group is commonly encountered in practice. DID removes bias from unobserved time-invariant confounders. While DID removes bias from time-invariant confounders, bias from time-varying confounders may be present. Hence, like any observational comparison, DID studies remain susceptible to bias from hidden confounders. Here, we develop a method of sensitivity analysis that allows investigators to quantify the amount of bias necessary to change a study's conclusions. Our method operates within a matched design that removes bias from observed baseline covariates. We develop methods for both binary and continuous outcomes. We then apply our methods to two different empirical examples from the social sciences. In the first application, we study the effect of changes to disability payments in Germany. In the second, we re-examine whether election day registration increased turnout in Wisconsin.
- Published
- 2019
29. Spillover Effects in Cluster Randomized Trials with Noncompliance
- Author
-
Kang, Hyunseung and Keele, Luke
- Subjects
Statistics - Methodology - Abstract
Cluster randomized trials (CRTs) are popular in public health and in the social sciences to evaluate a new treatment or policy where the new policy is randomly allocated to clusters of units rather than individual units. CRTs often feature both noncompliance, when individuals within a cluster are not exposed to the intervention, and individuals within a cluster may influence each other through treatment spillovers where those who comply with the new policy may affect the outcomes of those who do not. Here, we study the identification of causal effects in CRTs when both noncompliance and treatment spillovers are present. We prove that the standard analysis of CRT data with noncompliance using instrumental variables does not identify the usual complier average causal effect when treatment spillovers are present. We extend this result and show that no analysis of CRT data can unbiasedly estimate local network causal effects. Finally, we develop bounds for these causal effects under the assumption that the treatment is not harmful compared to the control. We demonstrate these results with an empirical study of a deworming intervention in Kenya.
- Published
- 2018
30. Extrapolating Treatment Effects in Multi-Cutoff Regression Discontinuity Designs
- Author
-
Cattaneo, Matias D., Keele, Luke, Titiunik, Rocio, and Vazquez-Bare, Gonzalo
- Subjects
Economics - Econometrics ,Statistics - Applications ,Statistics - Methodology - Abstract
In non-experimental settings, the Regression Discontinuity (RD) design is one of the most credible identification strategies for program evaluation and causal inference. However, RD treatment effect estimands are necessarily local, making statistical methods for the extrapolation of these effects a key area for development. We introduce a new method for extrapolation of RD effects that relies on the presence of multiple cutoffs, and is therefore design-based. Our approach employs an easy-to-interpret identifying assumption that mimics the idea of "common trends" in difference-in-differences designs. We illustrate our methods with data on a subsidized loan program on post-education attendance in Colombia, and offer new evidence on program effects for students with test scores away from the cutoff that determined program eligibility.
- Published
- 2018
31. Comparing Covariate Prioritization via Matching to Machine Learning Methods for Causal Inference using Five Empirical Applications
- Author
-
Keele, Luke and Small, Dylan
- Subjects
Statistics - Applications - Abstract
When investigators seek to estimate causal effects, they often assume that selection into treatment is based only on observed covariates. Under this identification strategy, analysts must adjust for observed confounders. While basic regression models have long been the dominant method of statistical adjustment, more robust methods based on matching or weighting have become more common. Of late, even more flexible methods based on machine learning methods have been developed for statistical adjustment. These machine learning methods are designed to be black box methods with little input from the researcher. Recent research used a data competition to evaluate various methods of statistical adjustment and found that black box methods out performed all other methods of statistical adjustment. Matching methods with covariate prioritization are designed for direct input from substantive investigators in direct contrast to black methods. In this article, we use a different research design to compare matching with covariate prioritization to black box methods. We use black box methods to replicate results from five studies where matching with covariate prioritization was used to customize the statistical adjustment in direct response to substantive expertise. We find little difference across the methods. We conclude with advice for investigators.
- Published
- 2018
32. Estimation Methods for Cluster Randomized Trials with Noncompliance: A Study of A Biometric Smartcard Payment System in India
- Author
-
Kang, Hyunseung and Keele, Luke
- Subjects
Statistics - Methodology - Abstract
Many policy evaluations occur in settings where treatment is randomized at the cluster level, and there is treatment noncompliance within each cluster. For example, villages might be assigned to treatment and control, but residents in each village may choose to comply or not with their assigned treatment status. When noncompliance is present, the instrumental variables framework can be used to identify and estimate causal effects. While a large literature exists on instrumental variables estimation methods, relatively little work has been focused on settings with clustered treatments. Here, we review extant methods for instrumental variable estimation in clustered designs and derive both the finite and asymptotic properties of these estimators. We prove that the properties of current estimators depend on unrealistic assumptions. We then develop a new IV estimation method for cluster randomized trials and study its formal properties. We prove that our IV estimator allows for possible treatment effect heterogeneity that is correlated with cluster size and is robust to low compliance rates within clusters. We evaluate these methods using simulations and apply them to data from a randomized intervention in India.
- Published
- 2018
33. Survivor-complier effects in the presence of selection on treatment, with application to a study of prompt ICU admission
- Author
-
Kennedy, Edward H., Harris, Steve, and Keele, Luke J.
- Subjects
Statistics - Methodology - Abstract
Pre-treatment selection or censoring (`selection on treatment') can occur when two treatment levels are compared ignoring the third option of neither treatment, in `censoring by death' settings where treatment is only defined for those who survive long enough to receive it, or in general in studies where the treatment is only defined for a subset of the population. Unfortunately, the standard instrumental variable (IV) estimand is not defined in the presence of such selection, so we consider estimating a new survivor-complier causal effect. Although this effect is generally not identified under standard IV assumptions, it is possible to construct sharp bounds. We derive these bounds and give a corresponding data-driven sensitivity analysis, along with nonparametric yet efficient estimation methods. Importantly, our approach allows for high-dimensional confounding adjustment, and valid inference even after employing machine learning. Incorporating covariates can tighten bounds dramatically, especially when they are strong predictors of the selection process. We apply the methods in a UK cohort study of critical care patients to examine the mortality effects of prompt admission to the intensive care unit, using ICU bed availability as an instrument.
- Published
- 2017
34. Optimal multilevel matching using network flows: An application to a summer reading intervention
- Author
-
Pimentel, Samuel D, Page, Lindsay C, Lenard, Matthew, and Keele, Luke
- Subjects
Pediatric ,Statistics ,Econometrics ,Statistics & Probability - Abstract
Many observational studies of causal effects occur in settings with clustered treatment assignment. In studies of this type, treatment is applied to entire clusters of units. For example, an educational intervention might be administered to all the students in a school. We develop a matching algorithm for multilevel data based on a network flow algorithm. Earlier work on multilevel matching relied on integer programming, which allows for balance targeting on specific covariates but can be slow with larger data sets. Although we cannot directly specify minimal levels of balance for individual covariates, our algorithm is fast and scales easily to larger data sets. We apply this algorithm to assess a school-based intervention through which students in treated schools were exposed to a new reading program during summer school. In one variant of the algorithm, where we match both schools and students, we change the causal estimand through optimal subset matching to better maintain common support. In a second variant, we relax the common support assumption to preserve the causal estimand by only matching on schools. We find that the summer intervention does not appear to increase reading test scores. In a sensitivity analysis, however, we determine that an unobserved confounder could easily mask a larger treatment effect.
- Published
- 2018
35. Inference for Instrumental Variables: A Randomization Inference Approach
- Author
-
Kang, Hyunseung, Peck, Laura, and Keele, Luke
- Subjects
Statistics - Methodology ,Statistics - Applications - Abstract
The method of instrumental variables (IV) provides a framework to study causal effects in both randomized experiments with noncompliance and in observational studies where natural circumstances produce as-if random nudges to accept treatment. Traditionally, inference for IV relied on asymptotic approximations of the distribution of the Wald estimator or two-stage least squares, often with structural modeling assumptions and/or moment conditions. In this paper, we utilize the randomization inference approach to IV inference. First, we outline the exact method, which uses the randomized assignment of treatment in experiments as a basis for inference, but lacks a closed-form solution and may be computationally infeasible in many applications. We then provide an alternative to the exact method, the almost exact method, which is computationally feasible but retains the advantages of the exact method. We also review asymptotic methods of inference, including those associated with two-stage least squares, and analytically compare them to randomization inference methods. We also perform additional comparisons using a set of simulations. We conclude with three different applications from the social sciences.
- Published
- 2016
36. Association of Delayed Antimicrobial Therapy with One-Year Mortality in Pediatric Sepsis
- Author
-
Han, Moonjoo, Fitzgerald, Julie C, Balamuth, Fran, Keele, Luke, Alpern, Elizabeth R, Lavelle, Jane, Chilutti, Marianne, Grundmeier, Robert W, Nadkarni, Vinay M, Thomas, Neal J, and Weiss, Scott L
- Subjects
Biomedical and Clinical Sciences ,Clinical Sciences ,Sepsis ,Hematology ,Infectious Diseases ,Infection ,Inflammatory and immune system ,Good Health and Well Being ,Adolescent ,Anti-Bacterial Agents ,Child ,Child ,Preschool ,Female ,Hospital Mortality ,Humans ,Infant ,Intensive Care Units ,Intensive Care Units ,Pediatric ,Logistic Models ,Male ,Retrospective Studies ,Shock ,Septic ,Emergency & Critical Care Medicine ,Clinical sciences - Abstract
ObjectiveDelayed antimicrobial therapy in sepsis is associated with increased hospital mortality, but the impact of antimicrobial timing on long-term outcomes is unknown. We tested the hypothesis that hourly delays to antimicrobial therapy are associated with 1-year mortality in pediatric severe sepsis.DesignRetrospective observational study.SettingQuaternary academic pediatric intensive care unit (PICU) from February 1, 2012 to June 30, 2013.PatientsOne hundred sixty patients aged ≤21 years treated for severe sepsis.InterventionsNone.Measurements and main resultsWe tested the association of hourly delays from sepsis recognition to antimicrobial administration with 1-year mortality using multivariable Cox and logistic regression. Overall 1-year mortality was 24% (39 patients), of whom 46% died after index PICU discharge. Median time from sepsis recognition to antimicrobial therapy was 137 min (IQR 65-287). After adjusting for severity of illness and comorbid conditions, hourly delays up to 3 h were not associated with 1-year mortality. However, increased 1-year mortality was evident in patients who received antimicrobials ≤1 h (aOR 3.8, 95% CI 1.2, 11.7) or >3 h (aOR 3.5, 95% CI 1.3, 9.8) compared with patients who received antimicrobials within 1 to 3 h from sepsis recognition. For the subset of patients who survived index PICU admission, antimicrobial therapy ≤1 h was also associated with increased 1-year mortality (aOR 5.5, 95% CI 1.1, 27.4), while antimicrobial therapy >3 h was not associated with 1-year mortality (aOR 2.2, 95% CI 0.5, 11.0).ConclusionsHourly delays to antimicrobial therapy, up to 3 h, were not associated with 1-year mortality in pediatric severe sepsis in this study. The finding that antimicrobial therapy ≤1 h from sepsis recognition was associated with increased 1-year mortality should be regarded as hypothesis-generating for future studies.
- Published
- 2017
37. The Plateau Problem in the Heteroskedastic Probit Model
- Author
-
Freeman, Eric, Keele, Luke, Park, David, Salzman, Julia, and Weickert, Brendan
- Subjects
Statistics - Methodology - Abstract
In parameter determination for the heteroskedastic probit model, both in simulated data and in actual data, we observe a failure of traditional local search methods to converge consistently to a single parameter vector, in contrast to the typical situation for the regular probit model. We identify features of the heteroskedastic probit log likelihood function that we argue tend to lead to this failure, and suggest ways to amend the local search methods to remedy the problem.
- Published
- 2015
38. Variable-Ratio Matching with Fine Balance in a Study of Peer Health Exchange
- Author
-
Keele, Luke, Pimentel, Sam, and Yoon, Frank
- Subjects
Statistics - Methodology - Abstract
In observational studies of treatment effects, matched samples are created so treated and control groups are similar in terms of observable covariates. Traditionally such matched samples consist of matched pairs. If a pair match fails to make treated and control units sufficiently comparable, alternative forms of matching may be necessary. One general strategy to improve balance is to match a variable number of control units to each treated unit. A more tailored strategy is to adopt a fine balance constraint. Under a fine balance constraint, a nominal covariate is exactly balanced, but it does not require individually matched treated and control subjects for this variable. In the example, we seek to construct a matched sample for an ongoing evaluation of Peer Health Exchange, an intervention in schools designed to decrease risky health behaviors among youth. We find that an optimal pair match that minimizes distances between pairs creates a matched sample where balance is poor. Here we propose a method to allow for fine balance constraints when each treated unit is matched to a variable number of control units, which is not currently possible using existing matching algorithms. Our approach uses the entire number to first determine the optimal number of controls for each treated unit. For each strata of matched treated units, we can then apply a fine balance constraint. We then demonstrate that a matched sample for the evaluation of the Peer Health Exchange based on a variable number of controls and fine balance constraint is superior to simply using a variable-ratio match., Comment: 30 pages, 2 figures
- Published
- 2014
39. Optimal Multilevel Matching in Clustered Observational Studies: A Case Study of the Effectiveness of Private Schools Under a Large-Scale Voucher System
- Author
-
Keele, Luke and Zubizarreta, Jose R.
- Subjects
Statistics - Methodology - Abstract
A distinctive feature of a clustered observational study is its multilevel or nested data structure arising from the assignment of treatment, in a non-random manner, to groups or clusters of units or individuals. Examples are ubiquitous in the health and social sciences including patients in hospitals, employees in firms, and students in schools. What is the optimal matching strategy in a clustered observational study? At first thought, one might start by matching clusters of individuals and then, within matched clusters, continue by matching individuals. But as we discuss in this paper, the optimal strategy is the opposite: in typical applications, where the intracluster correlation is not perfect, it is best to first match individuals and, once all possible combinations of matched individuals are known, then match clusters. In this paper we use dynamic and integer programming to implement this strategy and extend optimal matching methods to hierarchical and multilevel settings. Among other matched designs, our strategy can approximate a paired clustered randomized study by finding the largest sample of matched pairs of treated and control individuals within matched pairs of treated and control clusters that is balanced according to specifications given by the investigator. This strategy directly balances covariates both at the cluster and individual levels and does not require estimating the propensity score, although the propensity score can be balanced as an additional covariate. We illustrate our results with a case study of the comparative effectiveness of public versus private voucher schools in Chile, a question of intense policy debate in the country at the present.
- Published
- 2014
40. Comment : Will Competition-Winning Methods for Causal Inference Also Succeed in Practice?
- Author
-
Zhao, Qingyuan, Keele, Luke J., and Small, Dylan S.
- Published
- 2019
41. Identification, Inference and Sensitivity Analysis for Causal Mediation Effects
- Author
-
Imai, Kosuke, Keele, Luke, and Yamamoto, Teppei
- Subjects
Statistics - Methodology - Abstract
Causal mediation analysis is routinely conducted by applied researchers in a variety of disciplines. The goal of such an analysis is to investigate alternative causal mechanisms by examining the roles of intermediate variables that lie in the causal paths between the treatment and outcome variables. In this paper we first prove that under a particular version of sequential ignorability assumption, the average causal mediation effect (ACME) is nonparametrically identified. We compare our identification assumption with those proposed in the literature. Some practical implications of our identification result are also discussed. In particular, the popular estimator based on the linear structural equation model (LSEM) can be interpreted as an ACME estimator once additional parametric assumptions are made. We show that these assumptions can easily be relaxed within and outside of the LSEM framework and propose simple nonparametric estimation strategies. Second, and perhaps most importantly, we propose a new sensitivity analysis that can be easily implemented by applied researchers within the LSEM framework. Like the existing identifying assumptions, the proposed sequential ignorability assumption may be too strong in many applied settings. Thus, sensitivity analysis is essential in order to examine the robustness of empirical findings to the possible existence of an unmeasured confounder. Finally, we apply the proposed methods to a randomized experiment from political psychology. We also make easy-to-use software available to implement the proposed methods., Comment: Published in at http://dx.doi.org/10.1214/10-STS321 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)
- Published
- 2010
- Full Text
- View/download PDF
42. A Negative Correlation Strategy for Bracketing in Difference-in-Differences
- Author
-
Ye, Ting, primary, Keele, Luke, additional, Hasegawa, Raiden, additional, and Small, Dylan S., additional
- Published
- 2023
- Full Text
- View/download PDF
43. Social Capital and the Dynamics of Trust in Government
- Author
-
Keele, Luke
- Published
- 2007
44. Value Conflict and Volatility in Party Identification
- Author
-
Keele, Luke and Wolak, Jennifer
- Published
- 2006
45. The Authorities Really Do Matter: Party Control and Trust in Government
- Author
-
Keele, Luke
- Published
- 2005
- Full Text
- View/download PDF
46. Optimal Multilevel Matching in Clustered Observational Studies: A Case Study of the Effectiveness of Private Schools Under a Large-Scale Voucher System
- Author
-
Zubizarreta, José R. and Keele, Luke
- Published
- 2017
47. Instrumental variables: Don't throw the baby out with the bathwater
- Author
-
Keele, Luke and Small, Dylan
- Subjects
Instrumental variables -- Evaluation -- Quality management -- Design and construction ,Health ,Business ,Health care industry - Abstract
Soumerai and Koppel (1) make the case against the use of instrumental variables (IVs) in outcomes research. While they do not explicitly call for a halt to the use of [...]
- Published
- 2019
- Full Text
- View/download PDF
48. Comparison of Simulated Outcomes of Colorectal Cancer Surgery at the Highest-Performing vs Chosen Local Hospitals
- Author
-
Finn, Caitlin B., primary, Wirtalla, Chris, additional, Roberts, Sanford E., additional, Collier, Karole, additional, Mehta, Shivan J., additional, Guerra, Carmen E., additional, Airoldi, Edoardo, additional, Zhang, Xu, additional, Keele, Luke, additional, Aarons, Cary B., additional, Jensen, Shane T., additional, and Kelz, Rachel R., additional
- Published
- 2023
- Full Text
- View/download PDF
49. Clinical effectiveness and cost-effectiveness of emergency surgery for adult emergency hospital admissions with common acute gastrointestinal conditions: the ESORT study
- Author
-
Grieve, Richard, primary, Hutchings, Andrew, additional, Moler Zapata, Silvia, additional, O’Neill, Stephen, additional, Lugo-Palacios, David G, additional, Silverwood, Richard, additional, Cromwell, David, additional, Kircheis, Tommaso, additional, Silver, Elizabeth, additional, Snowdon, Claire, additional, Charlton, Paul, additional, Bellingan, Geoff, additional, Moonesinghe, Ramani, additional, Keele, Luke, additional, Smart, Neil, additional, and Hinchliffe, Robert, additional
- Published
- 2023
- Full Text
- View/download PDF
50. Generalized Structural Mean Models for Evaluating Depression as a Post-treatment Effect Modifier of a Jobs Training Intervention
- Author
-
Stephens Alisa, Keele Luke, and Joffe Marshall
- Subjects
post-treatment interaction ,structural mean model ,causal odds-ratio ,Mathematics ,QA1-939 ,Probabilities. Mathematical statistics ,QA273-280 - Abstract
In randomized controlled trials, the evaluation of an overall treatment effect is often followed by effect modification or subgroup analyses, where the possibility of a different magnitude or direction of effect for varying values of a covariate is explored. While studies of effect modification are typically restricted to pretreatment covariates, longitudinal experimental designs permit the examination of treatment effect modification by intermediate outcomes, where intermediates are measured after treatment but before the final outcome. We present a novel application of generalized structural mean models (GSMMs) for simultaneously assessing effect modification by post-treatment covariates and accounting for noncompliance to assigned treatment status. The proposed approach may also be used to identify post-treatment effect modifiers in the absence of noncompliance. The methods are evaluated using a simulation study that demonstrates that our approach retains consistent estimation of effect modification by intermediate variables that are affected by treatment and also predict outcomes. We illustrate the method using a randomized trial designed to promote re-employment through teaching skills to enhance self-esteem and inoculate job seekers against setbacks in the job search process. Our analysis provides some evidence that the intervention was much less successful among subjects that displayed higher levels of depression at intermediate post-treatment waves of the study. We also compare the assumptions of our approach and principal stratification as alternatives to account for differences in effects by intermediate variables.
- Published
- 2016
- Full Text
- View/download PDF
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.