Search

Objective Health care providers increasingly rely upon predictive algorithms when making important treatment decisions, however, evidence indicates that these tools can lead to inequitable outcomes across racial and socio-economic groups. In this study, we introduce a bias evaluation checklist that allows model developers and health care providers a means to systematically appraise a model’s potential to introduce bias. Materials and Methods Our methods include developing a bias evaluation checklist, a scoping literature review to identify 30-day hospital readmission prediction models, and assessing the selected models using the checklist. Results We selected 4 models for evaluation: LACE, HOSPITAL, Johns Hopkins ACG, and HATRIX. Our assessment identified critical ways in which these algorithms can perpetuate health care inequalities. We found that LACE and HOSPITAL have the greatest potential for introducing bias, Johns Hopkins ACG has the most areas of uncertainty, and HATRIX has the fewest causes for concern. Discussion Our approach gives model developers and health care providers a practical and systematic method for evaluating bias in predictive models. Traditional bias identification methods do not elucidate sources of bias and are thus insufficient for mitigation efforts. With our checklist, bias can be addressed and eliminated before a model is fully developed or deployed. Conclusion The potential for algorithms to perpetuate biased outcomes is not isolated to readmission prediction models; rather, we believe our results have implications for predictive models across health care. We offer a systematic method for evaluating potential bias with sufficient flexibility to be utilized across models and applications.

The use of artificial intelligence (AI) to help diagnose and manage disease is of increasing interest to researchers and clinicians. Volumes of health data are generated from smartphones and ubiquitous inexpensive sensors. By using these data, AI can offer otherwise unobtainable insights about disease burden and patient status in a free-living environment. Moreover, from clinical datasets AI can improve patient symptom monitoring and global epidemiologic efforts. While these applications are exciting, it is necessary to examine both the utility and limitations of these novel analytic methods. The most promising uses of AI remain aspirational. For example, defining the molecular subtypes of Parkinson’s disease will be assisted by future applications of AI to relevant datasets. This will allow clinicians to match patients to molecular therapies and will thus help launch precision medicine. Until AI proves its potential in pushing the frontier of precision medicine, its utility will primarily remain in individualized monitoring, complementing but not replacing movement disorders specialists.

ObjectiveTo characterize the longitudinal trajectory of estimated glomerular filtration rate (eGFR) in patients with systemic lupus erythematosus (SLE) and identify predictors of the change in eGFR trajectory.MethodsThe longitudinal eGFR levels of patients in the Hopkins Lupus Cohort were modelled by piecewise linear regression to evaluate the slope of different line segments. The slopes were classified into declining (≤−4 mL/min/1.73 m2per year), stable (−4 to 4 mL/min/1.73 m2per year), and increasing (≥4 mL/min/1.73 m2per year) states. The transition rate between states and the impact of clinical parameters were estimated by a Markov model.ResultsThe analysis was based on 494 SLE patients. At a mean follow-up of 8.8 years, 347 (70.2%), 107 (21.7%), 33 (6.7%), and 7 (1.4%) patients had zero, one, two, and three state transitions, respectively. In patients with no transition, 37 (10.7%), 308 (88.8%), and 2 (0.6%) were in declining, stable, and increasing state, respectively. In patients with one transition, 43 (40.2%) changed from declining to stable state while 29 (27.1%) changed from stable to declining state. When patients were in a non-declining GFR state, those who were younger and African Americans were more likely to transition to a declining GFR state. In adjusted analyses, high blood pressure, C4 and low hematocrit were associated with change from non-declining to declining state. High urine protein-to-creatinine ratio also tended to be associated with change from non-declining to declining state. African American patients were less likely to move from declining to non-declining state. Use of prednisone was associated with change from declining to non-declining state.ConclusionsPatients with high blood pressure, low complement C4, low haematocrit, and high urine protein-to-creatinine ratio are more likely to have a declining eGFR trajectory, while the use of prednisone stabilizes the declining eGFR trajectory.

Most studies on the association of in utero exposure to cigarette smoking and childhood overweight or obesity (OWO) were based on maternal self-reported smoking data and few were based on objective biomarkers. In this study, we evaluated the associations between self-reported and biomarkers of in utero exposure to cigarette smoking with risk of childhood OWO. We analyzed data from 2351 mother-child pairs in the Boston Birth Cohort, a US low-income minority cohort that enrolled children at birth and followed prospectively up to age 18 years. In utero smoking exposure was measured by maternal self-report and by maternal and cord plasma cotinine and hydroxycotinine metabolites. We assessed the individual and joint associations of each smoking exposure measure and maternal OWO with childhood OWO using multinomial logistic regressions. We used nested logistic regressions to investigate the childhood OWO prediction performance when adding maternal and cord plasma biomarkers as input covariates on top of self-reported data. Our results demonstrated that in utero cigarette smoking exposure defined by self-report and by maternal or cord metabolites were consistently associated with increased risk of long-term child OWO. Adding maternal and cord plasma biomarker information to self-reported data improved the prediction accuracy of long-term child OWO risk.

Precision nutrition is an emerging concept that aims to develop nutrition recommendations tailored to different people's circumstances and biological characteristics. Responses to dietary change and the resulting health outcomes from consuming different diets may vary significantly between people based on interactions between their genetic backgrounds, physiology, microbiome, underlying health status, behaviors, social influences, and environmental exposures. On 11-12 January 2021, the National Institutes of Health convened a workshop entitled "Precision Nutrition: Research Gaps and Opportunities" to bring together experts to discuss the issues involved in better understanding and addressing precision nutrition. The workshop proceeded in 3 parts: part I covered many aspects of genetics and physiology that mediate the links between nutrient intake and health conditions such as cardiovascular disease, Alzheimer disease, and cancer; part II reviewed potential contributors to interindividual variability in dietary exposures and responses such as baseline nutritional status, circadian rhythm/sleep, environmental exposures, sensory properties of food, stress, inflammation, and the social determinants of health; part III presented the need for systems approaches, with new methods and technologies that can facilitate the study and implementation of precision nutrition, and workforce development needed to create a new generation of researchers. The workshop concluded that much research will be needed before more precise nutrition recommendations can be achieved. This includes better understanding and accounting for variables such as age, sex, ethnicity, medical history, genetics, and social and environmental factors. The advent of new methods and technologies and the availability of considerably more data bring tremendous opportunity. However, the field must proceed with appropriate levels of caution and make sure the factors listed above are all considered, and systems approaches and methods are incorporated. It will be important to develop and train an expanded workforce with the goal of reducing health disparities and improving precision nutritional advice for all Americans.

Machine learning-based clinical decision support tools for sepsis create opportunities to identify at-risk patients and initiate treatments at early time points, which is critical for improving sepsis outcomes. In view of the increasing use of such systems, better understanding of how they are adopted and used by healthcare providers is needed. Here, we analyzed provider interactions with a sepsis early detection tool (Targeted Real-time Early Warning System), which was deployed at five hospitals over a 2-year period. Among 9,805 retrospectively identified sepsis cases, the early detection tool achieved high sensitivity (82% of sepsis cases were identified) and a high rate of adoption: 89% of all alerts by the system were evaluated by a physician or advanced practice provider and 38% of evaluated alerts were confirmed by a provider. Adjusting for patient presentation and severity, patients with sepsis whose alert was confirmed by a provider within 3 h had a 1.85-h (95% CI 1.66-2.00) reduction in median time to first antibiotic order compared to patients with sepsis whose alert was either dismissed, confirmed more than 3 h after the alert or never addressed in the system. Finally, we found that emergency department providers and providers who had previous interactions with an alert were more likely to interact with alerts, as well as to confirm alerts on retrospectively identified patients with sepsis. Beyond efforts to improve the performance of early warning systems, efforts to improve adoption are essential to their clinical impact and should focus on understanding providers' knowledge of, experience with and attitudes toward such systems.

Early recognition and treatment of sepsis are linked to improved patient outcomes. Machine learning-based early warning systems may reduce the time to recognition, but few systems have undergone clinical evaluation. In this prospective, multi-site cohort study, we examined the association between patient outcomes and provider interaction with a deployed sepsis alert system called the Targeted Real-time Early Warning System (TREWS). During the study, 590,736 patients were monitored by TREWS across five hospitals. We focused our analysis on 6,877 patients with sepsis who were identified by the alert before initiation of antibiotic therapy. Adjusting for patient presentation and severity, patients in this group whose alert was confirmed by a provider within 3 h of the alert had a reduced in-hospital mortality rate (3.3%, confidence interval (CI) 1.7, 5.1%, adjusted absolute reduction, and 18.7%, CI 9.4, 27.0%, adjusted relative reduction), organ failure and length of stay compared with patients whose alert was not confirmed by a provider within 3 h. Improvements in mortality rate (4.5%, CI 0.8, 8.3%, adjusted absolute reduction) and organ failure were larger among those patients who were additionally flagged as high risk. Our findings indicate that early warning systems have the potential to identify sepsis patients early and improve patient outcomes and that sepsis patients who would benefit the most from early treatment can be identified and prioritized at the time of the alert.

Dataset Shift in Clinical Trials This letter outlines how to identify, and potentially mitigate, common sources of “dataset shift” in machine-learning systems. This occurs when the model “training ...

Machine learning-based clinical decision support tools for sepsis create opportunities to identify at-risk patients and initiate treatments earlier, an important step in improving sepsis outcomes. Increasing use of such systems means quantifying and understanding provider adoption is critical. Using real-time provider interactions with a sepsis early detection tool (Targeted Real-time Early Warning System) deployed at five hospitals over a two-year period (469,419 screened patient encounters, 9,805 (2.1%) of which were retrospectively identified as having sepsis), we found high adoption rates (89% of alerts were evaluated by a physician or advanced practice provider) and an association between use of the tool and earlier treatment of sepsis patients (1.85 (95% CI: 1.66 - 2.00) hour reduction in median time to first antibiotics order). Further, we found that provider-related factors had the strongest association with alert adoption and that case complexity and atypical presentation were associated with dismissal of alerts on sepsis patients. Beyond improving the performance of the system, efforts to improve adoption should focus on provider knowledge, experience, and perceptions of the system.

Supplemental Digital Content is available in the text., OBJECTIVES: To evaluate factors predictive of clinical progression among coronavirus disease 2019 patients following admission, and whether continuous, automated assessments of patient status may contribute to optimal monitoring and management. DESIGN: Retrospective cohort for algorithm training, testing, and validation. SETTING: Eight hospitals across two geographically distinct regions. PATIENTS: Two-thousand fifteen hospitalized coronavirus disease 2019–positive patients. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Anticipating Respiratory failure in Coronavirus disease (ARC), a clinically interpretable, continuously monitoring prognostic model of acute respiratory failure in hospitalized coronavirus disease 2019 patients, was developed and validated. An analysis of the most important clinical predictors aligns with key risk factors identified by other investigators but contributes new insights regarding the time at which key factors first begin to exhibit aberrency and distinguishes features predictive of acute respiratory failure in coronavirus disease 2019 versus pneumonia caused by other types of infection. Departing from prior work, ARC was designed to update continuously over time as new observations (vitals and laboratory test results) are recorded in the electronic health record. Validation against data from two geographically distinct health systems showed that the proposed model achieved 75% specificity and 77% sensitivity and predicted acute respiratory failure at a median time of 32 hours prior to onset. Over 80% of true-positive alerts occurred in non-ICU settings. CONCLUSIONS: Patients admitted to non-ICU environments with coronavirus disease 2019 are at ongoing risk of clinical progression to severe disease, yet it is challenging to anticipate which patients will develop acute respiratory failure. A continuously monitoring prognostic model has potential to facilitate anticipatory rather than reactive approaches to escalation of care (e.g., earlier initiation of treatments for severe disease or structured monitoring and therapeutic interventions for high-risk patients).

Chronic disease now affects approximately half of the US population, causes 7 in 10 deaths, and accounts for roughly 80% of US health care expenditure. Because the root causes of chronic diseases are largely behavioral, effective therapies require frequent, individualized interventions that extend beyond the hospital and clinic to reach patients in their day-to-day lives. However, a mismatch currently exists between what the health care system is equipped to provide and the interventions necessary to effectively address the chronic disease burden. To remedy this health crisis, we present an individualized, data-driven digital approach for chronic disease management and prevention through precision behavior change. The rapid growth of information, biological, and communication technologies makes this an opportune time to develop digital tools that deliver precision interventions for health behavior change to address the chronic disease crisis. Building on this rapid growth, we propose a framework that includes the precise targeting of risk-producing behaviors using real-time sensing technology, machine learning data analysis to identify the most effective intervention, and delivery of that intervention with health-reinforcing feedback to provide real-time, individualized support to empower sustainable health behavior change.

The assessment of health and disease requires a set of criteria to define health status and progression. These health measures are referred to as “endpoints.” A “digital endpoint” is defined by its use of sensor-generated data often collected outside of a clinical setting such as in a patient’s free-living environment. Applicable sensors exist in an array of devices and can be applied in a diverse set of contexts. For example, a smartphone’s microphone might be used to diagnose or predict mild cognitive impairment due to Alzheimer’s disease or a wrist-worn activity monitor (such as those found in smartwatches) may be used to measure a drug’s effect on the nocturnal activity of patients with sickle cell disease. Digital endpoints are generating considerable excitement because they permit a more authentic assessment of the patient’s experience, reveal formerly untold realities of disease burden, and can cut drug discovery costs in half. However, before these benefits can be realized, effort must be applied not only to the technical creation of digital endpoints but also to the environment that allows for their development and application. The future of digital endpoints rests on meaningful interdisciplinary collaboration, sufficient evidence that digital endpoints can realize their promise, and the development of an ecosystem in which the vast quantities of data that digital endpoints generate can be analyzed. The fundamental nature of health care is changing. With coronavirus disease 2019 serving as a catalyst, there has been a rapid expansion of home care models, telehealth, and remote patient monitoring. The increasing adoption of these health-care innovations will expedite the requirement for a digital characterization of clinical status as current assessment tools often rely upon direct interaction with patients and thus are not fit for purpose to be administered remotely. With the ubiquity of relatively inexpensive sensors, digital endpoints are positioned to drive this consequential change. It is therefore not surprising that regulators, physicians, researchers, and consultants have each offered their assessment of these novel tools. However, as we further describe later, the broad adoption of digital endpoints will require a cooperative effort. In this article, we present an analysis of the current state of digital endpoints. We also attempt to unify the perspectives of the parties involved in the development and deployment of these tools. We conclude with an interdependent list of challenges that must be collaboratively addressed before these endpoints are widely adopted.

Key Points Question Can the risk of iatrogenic hypoglycemia resulting from insulin or insulin secretagogues be predicted continuously throughout hospitalization without the use of continuous glucose monitors? Findings In this cohort study of 54 978 admissions in a large US health care system, a stochastic gradient boosting machine learning model using 43 static and time-varying clinical predictors available in the electronic medical record accurately predicted the risk of iatrogenic hypoglycemia in a prediction horizon of 24 hours from the time of each point-of-care and serum glucose measurement throughout a patient’s hospital admission. Meaning These findings suggest that translating this machine learning prediction model into a real-time informatics alert embedded in the electronic medical record has the potential to reduce the incidence of iatrogenic hypoglycemia, a serious adverse event., Importance Accurate clinical decision support tools are needed to identify patients at risk for iatrogenic hypoglycemia, a potentially serious adverse event, throughout hospitalization. Objective To predict the risk of iatrogenic hypoglycemia within 24 hours after each blood glucose (BG) measurement during hospitalization using a machine learning model. Design, Setting, and Participants This retrospective cohort study, conducted at 5 hospitals within the Johns Hopkins Health System, included 54 978 admissions of 35 147 inpatients who had at least 4 BG measurements and received at least 1 U of insulin during hospitalization between December 1, 2014, and July 31, 2018. Data from the largest hospital were split into a 70% training set and 30% test set. A stochastic gradient boosting machine learning model was developed using the training set and validated on internal and external validation. Exposures A total of 43 clinical predictors of iatrogenic hypoglycemia were extracted from the electronic medical record, including demographic characteristics, diagnoses, procedures, laboratory data, medications, orders, anthropomorphometric data, and vital signs. Main Outcomes and Measures Iatrogenic hypoglycemia was defined as a BG measurement less than or equal to 70 mg/dL occurring within the pharmacologic duration of action of administered insulin, sulfonylurea, or meglitinide. Results This cohort study included 54 978 admissions (35 147 inpatients; median [interquartile range] age, 66.0 [56.0-75.0] years; 27 781 [50.5%] male; 30 429 [55.3%] White) from 5 hospitals. Of 1 612 425 index BG measurements, 50 354 (3.1%) were followed by iatrogenic hypoglycemia in the subsequent 24 hours. On internal validation, the model achieved a C statistic of 0.90 (95% CI, 0.89-0.90), a positive predictive value of 0.09 (95% CI, 0.08-0.09), a positive likelihood ratio of 4.67 (95% CI, 4.59-4.74), a negative predictive value of 1.00 (95% CI, 1.00-1.00), and a negative likelihood ratio of 0.22 (95% CI, 0.21-0.23). On external validation, the model achieved C statistics ranging from 0.86 to 0.88, positive predictive values ranging from 0.12 to 0.13, negative predictive values of 0.99, positive likelihood ratios ranging from 3.09 to 3.89, and negative likelihood ratios ranging from 0.23 to 0.25. Basal insulin dose, coefficient of variation of BG, and previous hypoglycemic episodes were the strongest predictors. Conclusions and Relevance These findings suggest that iatrogenic hypoglycemia can be predicted in a short-term prediction horizon after each BG measurement during hospitalization. Further studies are needed to translate this model into a real-time informatics alert and evaluate its effectiveness in reducing the incidence of inpatient iatrogenic hypoglycemia., This cohort study of hospitalized patients uses a machine learning model to predict the risk of iatrogenic hypoglycemia within 24 hours after blood glucose measurement during hospitalization.

Our objective in this study was to determine if machine learning (ML) can automatically recognize neonatal manipulations, along with associated changes in physiological parameters. A retrospective observational study was carried out in two Neonatal Intensive Care Units (NICUs) between December 2019 to April 2020. Both the video and physiological data (heart rate (HR) and oxygen saturation (SpO2)) were captured during NICU hospitalization. The proposed classification of neonatal manipulations was achieved by a deep learning system consisting of an Inception-v3 convolutional neural network (CNN), followed by transfer learning layers of Long Short-Term Memory (LSTM). Physiological signals prior to manipulations (baseline) were compared to during and after manipulations. The validation of the system was done using the leave-one-out strategy with input of 8 s of video exhibiting manipulation activity. Ten neonates were video recorded during an average length of stay of 24.5 days. Each neonate had an average of 528 manipulations during their NICU hospitalization, with the average duration of performing these manipulations varying from 28.9 s for patting, 45.5 s for a diaper change, and 108.9 s for tube feeding. The accuracy of the system was 95% for training and 85% for the validation dataset. In neonates &lt, 32 weeks&rsquo, gestation, diaper changes were associated with significant changes in HR and SpO2, and, for neonates &ge, gestation, patting and tube feeding were associated with significant changes in HR. The presented system can classify and document the manipulations with high accuracy. Moreover, the study suggests that manipulations impact physiological parameters.

Objective: Despite technological and treatment advancements over the past two decades, cardiogenic shock (CS) mortality has remained between 40-60%. A number of factors can lead to delayed diagnosis of CS, including gradual onset and nonspecific symptoms. Our objective was to develop an algorithm that can continuously monitor heart failure patients, and partition them into cohorts of high- and low-risk for CS.Methods: We retrospectively studied 24,461 patients hospitalized with acute decompensated heart failure, 265 of whom developed CS, in the Johns Hopkins Healthcare system. Our cohort identification approach is based on logistic regression, and makes use of vital signs, lab values, and medication administrations recorded during the normal course of care. Results: Our algorithm identified patients at high-risk of CS. Patients in the high-risk cohort had 10.2 times (95% confidence interval 6.1-17.2) higher prevalence of CS than those in the low-risk cohort. Patients who experienced cardiogenic shock while in the high-risk cohort were first deemed high-risk a median of 1.7 days (interquartile range 0.8 to 4.6) before cardiogenic shock diagnosis was made by their clinical team. Conclusions: This risk model was able to predict patients at higher risk of CS in a time frame that allowed a change in clinical care. Future studies need to evaluate if CS analysis of high-risk cohort identification may affect outcomes.

Objectives The objectives of this study are to construct the high definition phenotype (HDP), a novel time-series data structure composed of both primary and derived parameters, using heterogeneous clinical sources and to determine whether different predictive models can utilize the HDP in the neonatal intensive care unit (NICU) to improve neonatal mortality prediction in clinical settings. Materials and Methods A total of 49 primary data parameters were collected from July 2018 to May 2020 from eight level-III NICUs. From a total of 1546 patients, 757 patients were found to contain sufficient fixed, intermittent, and continuous data to create HDPs. Two different predictive models utilizing the HDP, one a logistic regression model (LRM) and the other a deep learning long–short-term memory (LSTM) model, were constructed to predict neonatal mortality at multiple time points during the patient hospitalization. The results were compared with previous illness severity scores, including SNAPPE, SNAPPE-II, CRIB, and CRIB-II. Results A HDP matrix, including 12 221 536 minutes of patient stay in NICU, was constructed. The LRM model and the LSTM model performed better than existing neonatal illness severity scores in predicting mortality using the area under the receiver operating characteristic curve (AUC) metric. An ablation study showed that utilizing continuous parameters alone results in an AUC score of >80% for both LRM and LSTM, but combining fixed, intermittent, and continuous parameters in the HDP results in scores >85%. The probability of mortality predictive score has recall and precision of 0.88 and 0.77 for the LRM and 0.97 and 0.85 for the LSTM. Conclusions and Relevance The HDP data structure supports multiple analytic techniques, including the statistical LRM approach and the machine learning LSTM approach used in this study. LRM and LSTM predictive models of neonatal mortality utilizing the HDP performed better than existing neonatal illness severity scores. Further research is necessary to create HDP–based clinical decision tools to detect the early onset of neonatal morbidities.

Here we present the MI-CLAIM checklist, a tool intended to improve transparent reporting of AI algorithms in medicine.

OBJECTIVES To evaluate associations between a readily availvable composite measurement of neighborhood socioeconomic disadvantage (the area deprivation index) and 30-day readmissions for patients who were previously hospitalized with sepsis. DESIGN A retrospective study. SETTING An urban, academic medical institution. PATIENTS The authors conducted a manual audit for adult patients (18 yr old or older) discharged with an International Classification of Diseases, 10th edition code of sepsis during the 2017 fiscal year to confirm that they met SEP-3 criteria. INTERVENTIONS None. MEASUREMENTS AND MAIN RESULTS The area deprivation index is a publicly available composite score constructed from socioeconomic components (e.g., income, poverty, education, housing characteristics) based on census block level, where higher scores are associated with more disadvantaged areas (range, 1-100). Using discharge data from the hospital population health database, residential addresses were geocoded and linked to their respective area deprivation index. Patient characteristics, contextual-level variables, and readmissions were compared by t tests for continuous variables and Fisher exact test for categorical variables. The associations between readmissions and area deprivation index were explored using logistic regression models. A total of 647 patients had an International Classification of Diseases, 10th edition diagnosis code of sepsis. Of these 647, 116 (17.9%) either died in hospital or were discharged to hospice and were excluded from our analysis. Of the remaining 531 patients, the mean age was 61.0 years (± 17.6 yr), 281 were females (52.9%), and 164 (30.9%) were active smokers. The mean length of stay was 6.9 days (± 5.6 d) with the mean Sequential Organ Failure Assessment score 4.9 (± 2.5). The mean area deprivation index was 54.2 (± 23.8). The mean area deprivation index of patients who were readmitted was 62.5 (± 27.4), which was significantly larger than the area deprivation index of patients not readmitted (51.8 [± 22.2]) (p < 0.001). In adjusted logistic regression models, a greater area deprivation index was significantly associated with readmissions (β, 0.03; p < 0.001). CONCLUSIONS Patients who reside in more disadvantaged neighborhoods have a significantly higher risk for 30-day readmission following a hospitalization for sepsis. The insight provided by neighborhood disadvantage scores, such as the area deprivation index, may help to better understand how contextual-level socioeconomic status affects the burden of sepsis-related morbidity.

Phenotype is the set of observable traits of an organism or condition. While advances in genetics, imaging, and molecular biology have improved our understanding of the underlying biology of Parkinson's disease (PD), clinical pheno-typing of PD still relies primarily on history and physical examination. These subjective, episodic, categorical assessments are valuable for diagnosis and care but have left gaps in our understanding of the PD phenotype. Sensors can provide objective, continuous, real-world data about the PD clinical phenotype, increase our knowledge of its pathology, enhance evaluation of therapies, and ultimately, improve patient care. In this paper, we explore the concept of deep phenotyping-the comprehensive assessment of a condition using multiple clinical, biological, genetic, imaging, and sensor-based tools-for PD. We discuss the rationale for, outline current approaches to, identify benefits and limitations of, and consider future directions for deep clinical phenotyping.

Background This study estimated the extent and predictors of primary nonadherence (i.e., prescriptions made by physicians but not initiated by patients) to methotrexate and to biologics or tofacitinib in rheumatoid arthritis (RA) patients who were newly prescribed these medications. Methods Using administrative claims linked with electronic health records (EHRs) from multiple healthcare provider organizations in the USA, RA patients who received a new prescription for methotrexate or biologics/tofacitinib were identified from EHRs. Claims data were used to ascertain filling or administration status. A logistic regression model for predicting primary nonadherence was developed and tested in training and test samples. Predictors were selected based on clinical judgment and LASSO logistic regression. Results A total of 36.8% of patients newly prescribed methotrexate failed to initiate methotrexate within 2 months; 40.6% of patients newly prescribed biologics/tofacitinib failed to initiate within 3 months. Factors associated with methotrexate primary nonadherence included age, race, region, body mass index, count of active drug ingredients, and certain previously diagnosed and treated conditions at baseline. Factors associated with biologics/tofacitinib primary nonadherence included age, insurance, and certain previously treated conditions at baseline. The area under the receiver operating characteristic curve of the logistic regression model estimated in the training sample and applied to the independent test sample was 0.86 and 0.78 for predicting primary nonadherence to methotrexate and to biologics/tofacitinib, respectively. Conclusions This study confirmed that failure to initiate new prescriptions for methotrexate and biologics/tofacitinib was common in RA patients. It is feasible to predict patients at high risk of primary nonadherence to methotrexate and to biologics/tofacitinib and to target such patients for early interventions to promote adherence.

Interest in machine-learning applications within medicine has been growing, but few studies have progressed to deployment in patient care. We present a framework, context and ultimately guidelines for accelerating the translation of machine-learning-based interventions in health care. To be successful, translation will require a team of engaged stakeholders and a systematic process from beginning (problem formulation) to end (widespread deployment).

Machine learning can help personalized decision support by learning models to predict individual treatment effects (ITE). This work studies the reliability of prediction-based decision-making in a task of deciding which action $a$ to take for a target unit after observing its covariates $\tilde{x}$ and predicted outcomes $\hat{p}(\tilde{y} \mid \tilde{x}, a)$. An example case is personalized medicine and the decision of which treatment to give to a patient. A common problem when learning these models from observational data is imbalance, that is, difference in treated/control covariate distributions, which is known to increase the upper bound of the expected ITE estimation error. We propose to assess the decision-making reliability by estimating the ITE model's Type S error rate, which is the probability of the model inferring the sign of the treatment effect wrong. Furthermore, we use the estimated reliability as a criterion for active learning, in order to collect new (possibly expensive) observations, instead of making a forced choice based on unreliable predictions. We demonstrate the effectiveness of this decision-making aware active learning in two decision-making tasks: in simulated data with binary outcomes and in a medical dataset with synthetic and continuous treatment outcomes., Published in Proceedings of the 36th International Conference on Machine Learning (ICML) 2019. 15 pages (10 paper + 5 supplementary), 7 figures

Digital health technologies (smartphones, smartwatches, and other body-worn sensors) can act as novel tools to aid in the diagnosis and remote objective monitoring of an individual’s disease symptoms, both in clinical care and in research. Nonetheless, such digital health technologies have yet to widely demonstrate value in clinical research due to insufficient data interpretability and lack of regulatory acceptance. Metadata, i.e., data that accompany and describe the primary data, can be utilized to better understand the context of the sensor data and can assist in data management, data sharing, and subsequent data analysis. The need for data and metadata standards for digital health technologies has been raised in academic and industry research communities and has also been noted by regulatory authorities. Therefore, to address this unmet need, we here propose a metadata set that reflects regulatory guidelines and that can serve as a conceptual map to (1) inform researchers on the metadata they should collect in digital health studies, aiming to increase the interpretability and exchangeability of their data, and (2) direct standard development organizations on how to extend their existing standards to incorporate digital health technologies. The proposed metadata set is informed by existing standards pertaining to clinical trials and medical devices, in addition to existing schemas that have supported digital health technology studies. We illustrate this specifically in the context of Parkinson’s disease, as a model for a wide range of other chronic conditions for which remote monitoring would be useful in both care and science. We invite the scientific and clinical research communities to apply the proposed metadata set to ongoing and planned research. Where the proposed metadata fall short, we ask users to contribute to its ongoing revision so that an adequate degree of consensus can be maintained in a rapidly evolving technology landscape.

Recent interest in the external validity of prediction models (i.e., the problem of different train and test distributions, known as dataset shift) has produced many methods for finding predictive distributions that are invariant to dataset shifts and can be used for prediction in new, unseen environments. However, these methods consider different types of shifts and have been developed under disparate frameworks, making it difficult to theoretically analyze how solutions differ with respect to stability and accuracy. Taking a causal graphical view, we use a flexible graphical representation to express various types of dataset shifts. Given a known graph of the data generating process, we show that all invariant distributions correspond to a causal hierarchy of graphical operators which disable the edges in the graph that are responsible for the shifts. The hierarchy provides a common theoretical underpinning for understanding when and how stability to shifts can be achieved, and in what ways stable distributions can differ. We use it to establish conditions for minimax optimal performance across environments, and derive new algorithms that find optimal stable distributions. Using this new perspective, we empirically demonstrate that that there is a tradeoff between minimax and average performance., Comment: Published in the Journal of Causal Inference

Machine Learning Special Issue Guest Editors Suchi Saria, Atul Butte, and Aziz Sheikh cut through the hyperbole with an accessible and accurate portrayal of the forefront of machine learning in clinical translation.

A major goal of neonatal medicine is to identify neonates at highest risk for morbidity and mortality. Previously, we developed PhysiScore (Saria et al., 2010), a novel tool for preterm morbidity risk prediction. We now further define links between overall individual morbidity risk, specific neonatal morbidities, and placental pathologies.102 placentas, including 38 from multiple gestations, were available from the previously defined PhysiScore cohort (gestational age ≤ 34 weeks and birth weight ≤ 2000 g). Placentas were analyzed for gross and histologic variables including maternal malperfusion, amniotic fluid infection sequence, chronic inflammation, and fetal vascular obstruction. Risk as determined by PhysiScore and recorded neonatal morbidities were tested for statistical association with placental findings.In pair-wise correlations, respiratory distress syndrome, bronchopulmonary dysplasia, acute hemodynamic instability, post-hemorrhagic hydrocephalus, culture-positive sepsis, and necrotizing enterocolitis each significantly correlated with at least one placenta histology variable. Amniotic fluid infection sequence (p = 0.039), specifically the fetal inflammatory response (p = 0.017), correlated with higher PhysiScores (greater morbidity) but was not independent of gestational age and birth weight. In multivariate analyses correlating variables with all nine morbidities, gestational age (p 0.001), placental size10th percentile (p = 0.031), full thickness perivillous fibrin deposition (p = 0.001), and amniotic fluid infection sequence (umbilical arteritis, p = 0.031; ≥2 chorionic plate vessels with vasculitis, p = 0.0125), each were significant associations.Amniotic fluid infection sequence plays a significant role in neonatal morbidity. Less neonatal morbidity was observed in older and heavier infants and those with small placental size and full thickness perivillous fibrin deposition. The combined assessment of placental gross and histologic findings together with physiologic risk evaluation may allow more precise prediction of neonatal morbidity risk soon after delivery.

IEEE Intelligent Systems once again selected 10 young AI scientists as " AI's 10 to Watch." This acknowledgment and celebration not only recognizes these young scientists and makes a positive impact in their academic career but also promotes the community and cutting-edge AI research among next-generation AI researchers, the industry, and the general public alike. The contributions are "Collective Decision Making in Multi-Agent Systems," by Haris Aziz, "From Causal Inference and Data Fusion to an Automated Scientist," by Elias Bareinboim, "Language, Vision, and Social AI," by Yejin Choi, "Algorithms for Machine Learning," by Daniel Hsu, "Learning Agents," by Shivaram Kalyanakrishnan, "Strategy and Bounded Rationality," by Reshef Meir, ";A Reasoning Engine for Tailoring Healthcare to the Individual," by Suchi Saria, "Pushing the Limits of Knowledge Representation and Reasoning," by Gerardo I. Simari, "Better Group Decision Making," by Lirong Xia, and "Distributed Constraint Optimization," by William Yeoh.

Reinforcement learning is applied to two large databases of electronic health records for patients admitted to an intensive care unit to identify individualized treatment strategies for correcting hypotension in sepsis.

Although the global burden of chronic diseases is rapidly growing, there is currently a mismatch between therapies that the healthcare system is equipped to provide and the interventions demonstrated to address the root causes of these chronic diseases. Since the underlying causes of chronic diseases are substantially lifestyle related, effective therapies require frequent, individualized behavioral interventions, but the intensive care that is traditionally provided in an effort to change behavior does not scale to meet the population’s chronic disease demands. This paper presents an alternative, scalable, technology-enabled framework for behavior change: we propose a “digital therapy” approach that comprises 1. precise targeting of the risk-producing behavior using real-time sensing technology, 2. predictive data analysis to identify the most effective intervention, and 3. delivery of that intervention with health-reinforcing feedback.

In this extended abstract, we generalize active learning to tasks where a human has to choose which action a to take for a target after observing its covariates and predicted outcomes. An example case is personalized medicine and the decision of which treatment to give to a patient. We show that standard active learning, which is not aware of the final task, would be very inefficient, and we introduce a new problem of decision-making-aware active learning. We formulate the problem as finding the query with the highest information gain for the specific decision-making task, assuming a rational decision-maker. The problem can be solved particularly efficiently assuming an expert able to answer queries about counterfactuals. We demonstrate the effectiveness of the proposed method in a binary outcome decision-making task using simulated data, and in a continuous-valued outcome task on the medical dataset IHDP with synthetic treatment outcomes. The outcomes are predicted using Gaussian processes. &nbsp

IMPORTANCE: Current Parkinson disease (PD) measures are subjective, rater-dependent, and assessed in clinic. Smartphones can measure PD features, yet no smartphone-derived rating score exists to assess motor symptom severity in real-world settings. OBJECTIVES: To develop an objective measure of PD severity and test construct validity by evaluating the ability of the measure to capture intraday symptom fluctuations, correlate with current standard PD outcome measures, and respond to dopaminergic therapy. DESIGN, SETTING, AND PARTICIPANTS: This observational study assessed individuals with PD who remotely completed 5 tasks (voice, finger tapping, gait, balance, and reaction time) on the smartphone application. We used a novel machine-learning-based approach to generate a mobile Parkinson disease score (mPDS) that objectively weighs features derived from each smartphone activity (eg, stride length from the gait activity) and is scaled from 0 to 100 (where higher scores indicate greater severity). Individuals with and without PD additionally completed standard in-person assessments of PD with smartphone assessments during a period of 6 months. MAIN OUTCOMES AND MEASURES: Ability of the mPDS to detect intraday symptom fluctuations, the correlation between the mPDS and standard measures, and the ability of the mPDS to respond to dopaminergic medication. RESULTS: The mPDS was derived from 6148 smartphone activity assessments from 129 individuals (mean [SD] age, 58.7 [8.6] years; 56 [43.4%] women). Gait features contributed most to the total mPDS (33.4%). In addition, 23 individuals with PD (mean [SD] age, 64.6 [11.5] years; 11 [48%] women) and 17 without PD (mean [SD] age 54.2 [16.5] years; 12 [71%] women) completed in-clinic assessments. The mPDS detected symptom fluctuations with a mean (SD) intraday change of 13.9 (10.3) points on a scale of 0 to 100. The measure correlated well with the Movement Disorder Society Unified Parkinson Disease's Rating Scale total (r = 0.81; P < .001) and part III only (r = 0.88; P < .001), the Timed Up and Go assessment (r = 0.72; P = .002), and the Hoehn and Yahr stage (r = 0.91; P < .001). The mPDS improved by a mean (SD) of 16.3 (5.6) points in response to dopaminergic therapy. CONCLUSIONS AND RELEVANCE: Using a novel machine-learning approach, we created and demonstrated construct validity of an objective PD severity score derived from smartphone assessments. This score complements standard PD measures by providing frequent, objective, real-world assessments that could enhance clinical care and evaluation of novel therapeutics.

ObjectiveTo develop and validate a multivariable prediction model for insulin-associated hypoglycemia in non-critically ill hospitalized adults.Research design and methodsWe collected pharmacologic, demographic, laboratory, and diagnostic data from 128 657 inpatient days in which at least 1 unit of subcutaneous insulin was administered in the absence of intravenous insulin, total parenteral nutrition, or insulin pump use (index days). These data were used to develop multivariable prediction models for biochemical and clinically significant hypoglycemia (blood glucose (BG) of ≤70 mg/dL and ResultsUsing predictors of age, weight, admitting service, insulin doses, mean BG, nadir BG, BG coefficient of variation (CVBG), diet status, type 1 diabetes, type 2 diabetes, acute kidney injury, chronic kidney disease (CKD), liver disease, and digestive disease, our model achieved a c-statistic of 0.77 (95% CI 0.75 to 0.78), positive likelihood ratio (+LR) of 3.5 (95% CI 3.4 to 3.6) and negative likelihood ratio (−LR) of 0.32 (95% CI 0.30 to 0.35) for prediction of biochemical hypoglycemia. Using predictors of sex, weight, insulin doses, mean BG, nadir BG, CVBG, diet status, type 1 diabetes, type 2 diabetes, CKD stage, and steroid use, our model achieved a c-statistic of 0.80 (95% CI 0.78 to 0.82), +LR of 3.8 (95% CI 3.7 to 4.0) and −LR of 0.2 (95% CI 0.2 to 0.3) for prediction of clinically significant hypoglycemia.ConclusionsHospitalized patients at risk of insulin-associated hypoglycemia can be identified using validated prediction models, which may support the development of real-time preventive interventions.