Search

In this paper, we consider methods for generating draws of a binary random variable whose expectation conditional on covariates follows a logistic regression model with known covariate coefficients. We examine approximations for finding a "balancing intercept," that is, a value for the intercept of the logistic model that leads to a desired marginal expectation for the binary random variable. We show that a recently proposed analytical approximation can produce inaccurate results, especially when targeting more extreme marginal expectations or when the linear predictor of the regression model has high variance. We then formulate the balancing intercept as a solution to an integral equation, implement a numerical approximation for solving the equation based on Monte Carlo methods, and show that the approximation works well in practice. Our approach to the basic problem of the balancing intercept provides an example of a broadly applicable strategy for formulating and solving problems that arise in the design of simulation studies used to evaluate or teach epidemiologic methods. [ABSTRACT FROM AUTHOR]

In diagnostic medicine, the true disease status of a patient is often represented on an ordinal scale—for example, cancer stage (0, I, II, III, or IV) or coronary artery disease severity measured using the Coronary Artery Disease Reporting and Data System (CAD-RADS) scale (none, minimal, mild, moderate, severe, or occluded). With advances in quantitation of diagnostic images and in artificial intelligence (AI), both supervised and unsupervised algorithms are being developed to help physicians correctly grade disease. Most of the diagnostic accuracy literature deals with binary disease status (disease present or absent); however, tests diagnosing ordinal-scaled diseases should not be reduced to a binary status just to simplify diagnostic accuracy testing. In this paper, we propose different characterizations of ordinal-scale accuracy for different clinical use scenarios, along with methods for comparing tests. In the simplest scenario, just the proportion of correct grades is considered; other scenarios address the magnitude and direction of misgrading; and at the other extreme, a weighted accuracy measure with weights based on the relative costs of different types of misgrading is presented. The various scenarios are illustrated using a coronary artery disease example where the accuracy of AI algorithms in providing patients with the correct CAD-RADS grade is assessed. [ABSTRACT FROM AUTHOR]

Researchers often face the problem of how to address missing data. Multiple imputation is a popular approach, with multiple imputation by chained equations (MICE) being among the most common and flexible methods for execution. MICE iteratively fits a predictive model for each variable with missing values, conditional on other variables in the data. In theory, any imputation model can be used to predict the missing values. However, if the predictive models are incorrectly specified, they may produce biased estimates of the imputed data, yielding inconsistent parameter estimates and invalid inference. Given the set of modeling choices that must be made in conducting multiple imputation, in this paper we propose a data-adaptive approach to model selection. Specifically, we adapt MICE to incorporate an ensemble algorithm, Super Learner, to predict the conditional mean for each missing value, and we also incorporate a local kernel-based estimate of variance. We present a set of simulations indicating that this approach produces final parameter estimates with lower bias and better coverage than other commonly used imputation methods. These results suggest that using a flexible machine learning imputation approach can be useful in settings where data are missing at random, especially when the relationships among the variables are complex., (© The Author(s) 2021. Published by Oxford University Press on behalf of the Johns Hopkins Bloomberg School of Public Health. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com.)

An increasing number of recent studies have suggested that doubly robust estimators with cross-fitting should be used when estimating causal effects with machine learning methods. However, not all existing programs that implement doubly robust estimators support machine learning methods and cross-fitting, or provide estimates on multiplicative scales. To address these needs, we developed AIPW , a software package implementing augmented inverse probability weighting (AIPW) estimation of average causal effects in R (R Foundation for Statistical Computing, Vienna, Austria). Key features of the AIPW package include cross-fitting and flexible covariate adjustment for observational studies and randomized controlled trials (RCTs). In this paper, we use a simulated RCT to illustrate implementation of the AIPW estimator. We also perform a simulation study to evaluate the performance of the AIPW package compared with other doubly robust implementations, including CausalGAM , npcausal , tmle , and tmle3. Our simulation showed that the AIPW package yields performance comparable to that of other programs. Furthermore, we also found that cross-fitting substantively decreases the bias and improves the confidence interval coverage for doubly robust estimators fitted with machine learning algorithms. Our findings suggest that the AIPW package can be a useful tool for estimating average causal effects with machine learning methods in RCTs and observational studies. [ABSTRACT FROM AUTHOR]

The authors respond to a commentary on a study which evaluated properties of machine learning methods for estimating causal effects. They point out the need for correct parametric regression model specification when using machine learning methods. They describe two possible scenarios when using stacked regression to estimate causal effects. They agree with conducting study-specific simulations to evaluate potential issues with the use of different algorithms in the stacked regression library.

Accurate forecasts can inform response to outbreaks. Most efforts in influenza forecasting have focused on predicting influenza-like activity, with fewer on influenza-related hospitalizations. We conducted a simulation study to evaluate a super learner's predictions of 3 seasonal measures of influenza hospitalizations in the United States: peak hospitalization rate, peak hospitalization week, and cumulative hospitalization rate. We trained an ensemble machine learning algorithm on 15,000 simulated hospitalization curves and generated weekly predictions. We compared the performance of the ensemble (weighted combination of predictions from multiple prediction algorithms), the best-performing individual prediction algorithm, and a naive prediction (median of a simulated outcome distribution). Ensemble predictions performed similarly to the naive predictions early in the season but consistently improved as the season progressed for all prediction targets. The best-performing prediction algorithm in each week typically had similar predictive accuracy compared with the ensemble, but the specific prediction algorithm selected varied by week. An ensemble super learner improved predictions of influenza-related hospitalizations, relative to a naive prediction. Future work should examine the super learner's performance using additional empirical data on influenza-related predictors (e.g. influenza-like illness). The algorithm should also be tailored to produce prospective probabilistic forecasts of selected prediction targets. [ABSTRACT FROM AUTHOR]

Patients with opioid use disorder (OUD) tend to get assigned to one of 3 medications based on the treatment program to which the patient presents (e.g. opioid treatment programs tend to treat patients with methadone, while office-based practices tend to prescribe buprenorphine). It is possible that optimally matching patients with treatment type would reduce the risk of return to regular opioid use (RROU). We analyzed data from 3 comparative effectiveness trials from the US National Institute on Drug Abuse Clinical Trials Network (CTN0027, 2006–2010; CTN0030, 2006–2009; and CTN0051 2014–2017), in which patients with OUD (n  = 1,459) were assigned to treatment with either injection extended-release naltrexone (XR-NTX), sublingual buprenorphine-naloxone (BUP-NX), or oral methadone. We learned an individualized rule by which to assign medication type such that risk of RROU during 12 weeks of treatment would be minimized, and then estimated the amount by which RROU risk could be reduced if the rule were applied. Applying our estimated treatment rule would reduce risk of RROU compared with treating everyone with methadone (relative risk (RR) = 0.79, 95% confidence interval (CI): 0.60, 0.97) or treating everyone with XR-NTX (RR = 0.71, 95% CI: 0.47, 0.96). Applying the estimated treatment rule would have resulted in a similar risk of RROU to that of with treating everyone with BUP-NX (RR = 0.92, 95% CI: 0.73, 1.11). [ABSTRACT FROM AUTHOR]

Dementia is a complex, progressive syndrome characterized by cognitive decline and disability. Gold-standard dementia diagnosis requires several hours of cognitive and clinical assessment and review by a panel of clinicians and is infeasible in large population-based cohort studies. Alternatively, algorithmic dementia classification methods, which use models that take into account measures of cognition and functional limitations or cognitive and functional-limitation score cutoffs, have been developed to predict dementia status for participants in large studies. Developing accurate dementia classification algorithms is crucial for high-quality studies of the distribution and determinants of dementia. The accompanying article by Nichols et al. (Am J Epidemiol. 2023;192(4):520–534) assesses differences in associations of measures of cognition and functional limitations with prevalent dementia versus incident dementia and discusses implications for algorithmic dementia classification in research studies. This work highlights important opportunities for tailoring measures of cognition and functional limitations to study goals by selecting optimal measures and developing and validating algorithms specific to study needs. Combining efficient, high-quality assessments of cognition and functional limitations with innovative study designs will facilitate collection of higher-quality measurements in larger samples and support future development of accurate dementia classifications, ultimately leading to more impactful epidemiologic studies. [ABSTRACT FROM AUTHOR]

We sought to determine whether machine learning and natural language processing (NLP) applied to electronic medical records could improve performance of automated health-care claims-based algorithms to identify anaphylaxis events using data on 516 patients with outpatient, emergency department, or inpatient anaphylaxis diagnosis codes during 2015–2019 in 2 integrated health-care institutions in the Northwest United States. We used one site's manually reviewed gold-standard outcomes data for model development and the other's for external validation based on cross-validated area under the receiver operating characteristic curve (AUC), positive predictive value (PPV), and sensitivity. In the development site 154 (64%) of 239 potential events met adjudication criteria for anaphylaxis compared with 180 (65%) of 277 in the validation site. Logistic regression models using only structured claims data achieved a cross-validated AUC of 0.58 (95% CI: 0.54, 0.63). Machine learning improved cross-validated AUC to 0.62 (0.58, 0.66); incorporating NLP-derived covariates further increased cross-validated AUCs to 0.70 (0.66, 0.75) in development and 0.67 (0.63, 0.71) in external validation data. A classification threshold with cross-validated PPV of 79% and cross-validated sensitivity of 66% in development data had cross-validated PPV of 78% and cross-validated sensitivity of 56% in external data. Machine learning and NLP-derived data improved identification of validated anaphylaxis events. [ABSTRACT FROM AUTHOR]

A correction to the article "Identifying Predictors of Opioid Overdose Death at a Neighborhood Level With Machine Learning" that was published in a 2022 issue is presented.

Dengue is a serious public health concern in Brazil and globally. In the absence of a universal vaccine or specific treatments, prevention relies on vector control and disease surveillance. Accurate and early forecasts can help reduce the spread of the disease. In this study, we developed a model for predicting monthly dengue cases in Brazilian cities 1 month ahead, using data from 2007–2019. We compared different machine learning algorithms and feature selection methods using epidemiologic and meteorological variables. We found that different models worked best in different cities, and a random forests model trained on monthly dengue cases performed best overall. It produced lower errors than a seasonal naive baseline model, gradient boosting regression, a feed-forward neural network, or support vector regression. For each city, we computed the mean absolute error between predictions and true monthly numbers of dengue cases on the test data set. The median error across all cities was 12.2 cases. This error was reduced to 11.9 when selecting the optimal combination of algorithm and input features for each city individually. Machine learning and especially decision tree ensemble models may contribute to dengue surveillance in Brazil, as they produce low out-of-sample prediction errors for a geographically diverse set of cities. [ABSTRACT FROM AUTHOR]

The authors discuss a sampling algorithm for obtaining estimates from data such as those given by Google via the Google Trends Extended for Health (GT-E) API. Highlights include an outline of different samplings based on width of sampling range, a Visual Basic for Applications code sheet provided for the algorithm, and the use of the algorithm with the unscaled GT-E data in favor of the scaled public facing data.

Evaluating the impacts of population-level interventions (e.g. changes to state legislation) can be challenging as conducting randomized experiments is often impractical and inappropriate, especially in settings where the intervention is implemented in a single, aggregate unit (e.g. a country or state). A common nonrandomized alternative is to compare outcomes in the treated unit(s) with unexposed controls both before and after the intervention. However, the validity of these designs depends on the use of controls that closely resemble the treated unit on before-intervention characteristics and trends on the outcome, and suitable controls may be difficult to find because the number of potential control regions is typically limited. The synthetic control method provides a potential solution to these problems by using a data-driven algorithm to identify an optimal weighted control unit—a "synthetic control"—based on data from before the intervention from available control units. While popular in the social sciences, the method has not garnered as much attention in health research, perhaps due to a lack of accessible texts aimed at health researchers. We address this gap by providing a comprehensive, nontechnical tutorial on the synthetic control method, using a worked example evaluating Florida's "stand your ground" law to illustrate methodological and practical considerations. [ABSTRACT FROM AUTHOR]