Search

Estimating varying treatment effects in randomized trials with noncompliance is inherently challenging since variation comes from two separate sources: variation in the impact itself and variation in the compliance rate. In this setting, existing flexible machine learning methods are highly sensitive to the weak instruments problem, in which the compliance rate is (locally) close to zero. Our main methodological contribution is to present a Bayesian Causal Forest model for binary response variables in scenarios with noncompliance. By repeatedly imputing individuals' compliance types, we can flexibly estimate heterogeneous treatment effects among compliers. Simulation studies demonstrate the usefulness of our approach when compliance and treatment effects are heterogeneous. We apply the method to detect and analyze heterogeneity in the treatment effects in the Illinois Workplace Wellness Study, which not only features heterogeneous and one-sided compliance but also several binary outcomes of interest. We demonstrate the methodology on three outcomes one year after intervention. We confirm a null effect on the presence of a chronic condition, discover meaningful heterogeneity impact of the intervention on metabolic parameters though the average effect is null in classical partial effect estimates, and find substantial heterogeneity in individuals' perception of management prioritization of health and safety., Comment: 24 pages, 8 figures

We study the impact of teenage sports participation on early-adulthood health using longitudinal data from the National Study of Youth and Religion. We focus on two primary outcomes measured at ages 23--28 -- self-rated health and total score on the PHQ9 Patient Depression Questionnaire -- and control for several potential confounders related to demographics and family socioeconomic status. To probe the possibility that certain types of sports participation may have larger effects on health than others, we conduct a matched observational study at each level within a hierarchy of exposures. Our hierarchy ranges from broadly defined exposures (e.g., participation in any organized after-school activity) to narrow (e.g., participation in collision sports). We deployed an ordered testing approach that exploits the hierarchical relationships between our exposure definitions to perform our analyses while maintaining a fixed family-wise error rate. Compared to teenagers who did not participate in any after-school activities, those who participated in sports had statistically significantly better self-rated and mental health outcomes in early adulthood., Comment: The pre-analysis protocol for this study is available at arXiv:2211.02104

We introduce a three-step framework to determine at which pitches Major League batters should swing. Unlike traditional plate discipline metrics, which implicitly assume that all batters should always swing at (resp. take) pitches inside (resp. outside) the strike zone, our approach explicitly accounts not only for the players and umpires involved in the pitch but also in-game contextual information like the number of outs, the count, baserunners, and score. We first fit flexible Bayesian nonparametric models to estimate (i) the probability that the pitch is called a strike if the batter takes the pitch; (ii) the probability that the batter makes contact if he swings; and (iii) the number of runs the batting team is expected to score following each pitch outcome (e.g. swing and miss, take a called strike, etc.). We then combine these intermediate estimates to determine whether swinging increases the batting team's run expectancy. Our approach enables natural uncertainty propagation so that we can not only determine the optimal swing/take decision but also quantify our confidence in that decision. We illustrate our framework using a case study of pitches faced by Mike Trout in 2019.

We will study the impact of adolescent sports participation on early-adulthood health using longitudinal data from the National Study of Youth and Religion. We focus on two primary outcomes measured at ages 23--28 -- self-rated health and total score on the PHQ9 Patient Depression Questionnaire -- and control for several potential confounders related to demographics and family socioeconomic status. Comparing outcomes between sports participants and matched non-sports participants with similar confounders is straightforward. Unfortunately, an analysis based on such a broad exposure cannot probe the possibility that participation in certain types of sports (e.g. collision sports like football or soccer) may have larger effects on health than others. In this study, we introduce a hierarchy of exposure definitions, ranging from broad (participation in any after-school organized activity) to narrow (e.g. participation in limited-contact sports). We will perform separate matched observational studies, one for each definition, to estimate the health effects of several levels of sports participation. In order to conduct these studies while maintaining a fixed family-wise error rate, we developed an ordered testing approach that exploits the logical relationships between exposure definitions. Our study will also consider several secondary outcomes including body mass index, life satisfaction, and problematic drinking behavior.

We demonstrate how Hahn et al.'s Bayesian Causal Forests model (BCF) can be used to estimate conditional average treatment effects for the longitudinal dataset in the 2022 American Causal Inference Conference Data Challenge. Unfortunately, existing implementations of BCF do not scale to the size of the challenge data. Therefore, we developed flexBCF -- a more scalable and flexible implementation of BCF -- and used it in our challenge submission. We investigate the sensitivity of our results to the choice of propensity score estimation method and the use of sparsity-inducing regression tree priors. While we found that our overall point predictions were not especially sensitive to these modeling choices, we did observe that running BCF with flexibly estimated propensity scores often yielded better-calibrated uncertainty intervals.

As a baseball game progresses, batters appear to perform better the more times they face a particular pitcher. The apparent drop-off in pitcher performance from one time through the order to the next, known as the Time Through the Order Penalty (TTOP), is often attributed to within-game batter learning. Although the TTOP has largely been accepted within baseball and influences many managers' in-game decision making, we argue that existing approaches of estimating the size of the TTOP cannot disentangle continuous evolution in pitcher performance over the course of the game from discontinuities between successive times through the order. Using a Bayesian multinomial regression model, we find that, after adjusting for confounders like batter and pitcher quality, handedness, and home field advantage, there is little evidence of strong discontinuity in pitcher performance between times through the order. Our analysis suggests that the start of the third time through the order should not be viewed as a special cutoff point in deciding whether to pull a starting pitcher., Comment: Accepted to JQAS

Accurate estimation of the change in crime over time is a critical first step towards better understanding of public safety in large urban environments. Bayesian hierarchical modeling is a natural way to study spatial variation in urban crime dynamics at the neighborhood level, since it facilitates principled ``sharing of information'' between spatially adjacent neighborhoods. Typically, however, cities contain many physical and social boundaries that may manifest as spatial discontinuities in crime patterns. In this situation, standard prior choices often yield overly-smooth parameter estimates, which can ultimately produce mis-calibrated forecasts. To prevent potential over-smoothing, we introduce a prior that partitions the set of neighborhoods into several clusters and encourages spatial smoothness within each cluster. In terms of model implementation, conventional stochastic search techniques are computationally prohibitive, as they must traverse a combinatorially vast space of partitions. We introduce an ensemble optimization procedure that simultaneously identifies several high probability partitions by solving one optimization problem using a new local search strategy. We then use the identified partitions to estimate crime trends in Philadelphia between 2006 and 2017. On simulated and real data, our proposed method demonstrates good estimation and partition selection performance.

Using high-resolution player tracking data made available by the National Football League (NFL) for their 2019 Big Data Bowl competition, we introduce the Expected Hypothetical Completion Probability (EHCP), a objective framework for evaluating plays. At the heart of EHCP is the question "on a given passing play, did the quarterback throw the pass to the receiver who was most likely to catch it?" To answer this question, we first built a Bayesian non-parametric catch probability model that automatically accounts for complex interactions between inputs like the receiver's speed and distances to the ball and nearest defender. While building such a model is, in principle, straightforward, using it to reason about a hypothetical pass is challenging because many of the model inputs corresponding to a hypothetical are necessarily unobserved. To wit, it is impossible to observe how close an un-targeted receiver would be to his nearest defender had the pass been thrown to him instead of the receiver who was actually targeted. To overcome this fundamental difficulty, we propose imputing the unobservable inputs and averaging our model predictions across these imputations to derive EHCP. In this way, EHCP can track how the completion probability evolves for each receiver over the course of a play in a way that accounts for the uncertainty about missing inputs., Comment: This paper elaborates on work done for the NFL 2019 Big Data Bowl contest. Manuscript accepted at the Journal of Quantitative Analysis in Sports

American football is the most popular high school sport and is among the leading cause of injury among adolescents. While there has been considerable recent attention on the link between football and cognitive decline, there is also evidence of higher than expected rates of pain, obesity, and lower quality of life among former professional players, either as a result of repetitive head injury or through different mechanisms. Previously hidden downstream effects of playing football may have far-reaching public health implications for participants in youth and high school football programs. Our proposed study is a retrospective observational study that compares 1,153 high school males who played varsity football with 2,751 male students who did not. 1,951 of the control subjects did not play any sport and the remaining 800 controls played a non-contact sport. Our primary outcome is self-rated health measured at age 65. To control for potential confounders, we adjust for pre-exposure covariates with matching and model-based covariance adjustment. We will conduct an ordered testing procedure designed to use the full pool of 2,751 controls while also controlling for possible unmeasured differences between students who played sports and those who did not. We will quantitatively assess the sensitivity of the results to potential unmeasured confounding. The study will also assess secondary outcomes of pain, difficulty with activities of daily living, and obesity, as these are both important to individual well-being and have public health relevance.

More than 1 million students play high school American football annually, but many health professionals have recently questioned its safety or called for its ban. These concerns have been partially driven by reports of chronic traumatic encephalopathy (CTE), increased risks of neurodegenerative disease, and associations between concussion history and later-life cognitive impairment and depression among retired professional football players. A recent observational study of a cohort of men who graduated from a Wisconsin high school in 1957 found no statistically significant harmful effects of playing high school football on a range of cognitive, psychological, and socio-economic outcomes measured at ages 35, 54, 65, and 72. Unfortunately, these findings may not generalize to younger populations, thanks to changes and improvements in football helmet technology and training techniques. In particular, these changes may have led to increased perceptions of safety but ultimately more dangerous styles of play, characterized by the frequent sub-concussive impacts thought to be associated with later-life neurological decline. In this work, we replicate the methodology of that earlier matched observational study using data from the National Longitudinal Study of Adolescent to Adult Health (Add Health). These include adolescent and family co-morbidities, academic experience, self-reported levels of general health and physical activity, and the score on the Add Health Picture Vocabulary Test. Our primary outcome is the CES-D score measured in 2008 when subjects were aged 24 -- 34 and settling into early adulthood. We also examine several secondary outcomes related to physical and psychological health, including suicidality. Our results can provide insight into the natural history of potential football-related decline and dysfunction., Comment: Updated tables summarizing the matches constructed

A large body of work links traumatic brain injury (TBI) in adulthood to the onset of Alzheimer's disease (AD). AD is the chief cause of dementia, leading to reduced cognitive capacity and autonomy and increased mortality risk. More recently, researchers have sought to investigate whether TBI experienced in early-life may influence trajectories of cognitive dysfunction in adulthood. It has been speculated that early-life participation in collision sports may lead to poor cognitive and mental health outcomes. However, to date, the few studies to investigate this relationship have produced mixed results. We propose to extend this literature by conducting a prospective study on the effects of early-life participation in collision sports on later-life cognitive health using the Swedish Adoption/Twin Study on Aging (SATSA). The SATSA is unique in its sampling of monozygotic and dizygotic twins reared together (respectively MZT, DZT) and twins reared apart (respectively MZA, DZA). The proposed analysis is a prospective study of 660 individuals comprised of 270 twin pairs and 120 singletons. Seventy-eight (11.8% individuals reported participation in collision sports. Our primary outcome will be an indicator of cognitive impairment determined by scores on the Mini-Mental State Examination (MMSE). We will also consider several secondary cognitive outcomes including verbal and spatial ability, memory, and processing speed. Our sample will be restricted to individuals with at least one MMSE score out of seven repeated assessments spaced approximately three years apart. We will adjust for age, sex, and education in each of our models., Comment: Updated methodology and tables

Since the advent of high-resolution pitch tracking data (PITCHf/x), many in the sabermetrics community have attempted to quantify a Major League Baseball catcher's ability to "frame" a pitch (i.e. increase the chance that a pitch is called as a strike). Especially in the last three years, there has been an explosion of interest in the "art of pitch framing" in the popular press as well as signs that teams are considering framing when making roster decisions. We introduce a Bayesian hierarchical model to estimate each umpire's probability of calling a strike, adjusting for pitch participants, pitch location, and contextual information like the count. Using our model, we can estimate each catcher's effect on an umpire's chance of calling a strike.We are then able to translate these estimated effects into average runs saved across a season. We also introduce a new metric, analogous to Jensen, Shirley, and Wyner's Spatially Aggregate Fielding Evaluation metric, which provides a more honest assessment of the impact of framing.

A potential causal relationship between head injuries sustained by NFL players and later-life neurological decline may have broad implications for participants in youth and high school football programs. However, brain trauma risk at the professional level may be different than that at the youth and high school levels and the long-term effects of participation at these levels is as-yet unclear. To investigate the effect of playing high school football on later life depression and cognitive functioning, we propose a retrospective observational study using data from the Wisconsin Longitudinal Study (WLS) of graduates from Wisconsin high schools in 1957. We compare 1,153 high school males who played varsity football to 2,751 male students who did not. 1,951 of the control subjects did not play any sport and the remaining 800 controls played a non-contact sport. We focus on two primary outcomes measured at age 65: a composite cognitive outcome measuring verbal fluency and memory and the modified CES-D depression score. To control for potential confounders we adjust for pre-exposure covariates such as IQ with matching and model-based covariate adjustment. We will conduct an ordered testing procedure that uses all 2,751 controls while controlling for possible unmeasured differences between students who played sports and those who did not. We will quantitatively assess the sensitivity of the results to potential unmeasured confounding. The study will also consider several secondary outcomes of clinical interest such as aggression and heavy drinking. The rich set of pre-exposure variables, relatively unbiased sampling, and longitudinal nature of the WLS dataset make the proposed analysis unique among related studies that rely primarily on convenience samples of football players with reported neurological symptoms., Comment: Prior to performing the proposed analysis, we will register this pre-analysis plan on clincialtrials.gov

Traditional NBA player evaluation metrics are based on scoring differential or some pace-adjusted linear combination of box score statistics like points, rebounds, assists, etc. These measures treat performances with the outcome of the game still in question (e.g. tie score with five minutes left) in exactly the same way as they treat performances with the outcome virtually decided (e.g. when one team leads by 30 points with one minute left). Because they ignore the context in which players perform, these measures can result in misleading estimates of how players help their teams win. We instead use a win probability framework for evaluating the impact NBA players have on their teams' chances of winning. We propose a Bayesian linear regression model to estimate an individual player's impact, after controlling for the other players on the court. We introduce several posterior summaries to derive rank-orderings of players within their team and across the league. This allows us to identify highly paid players with low impact relative to their teammates, as well as players whose high impact is not captured by existing metrics., Comment: To appear in the Journal of Quantitative Analysis of Sport