Search

ObjectiveWe aimed to investigate the impact of social circumstances on cancer therapy selection using natural language processing to derive insights from social worker documentation.Materials and methodsWe developed and employed a Bidirectional Encoder Representations from Transformers (BERT) based approach, using a hierarchical multi-step BERT model (BERT-MS), to predict the prescription of targeted cancer therapy to patients based solely on documentation by clinical social workers. Our corpus included free-text clinical social work notes, combined with medication prescription information, for all patients treated for breast cancer at UCSF between 2012 and 2021. We conducted a feature importance analysis to identify the specific social circumstances that impact cancer therapy regimen.ResultsUsing only social work notes, we consistently predicted the administration of targeted therapies, suggesting systematic differences in treatment selection exist due to non-clinical factors. The findings were confirmed by several language models, with GatorTron achieving the best performance with an area under the receiver operating characteristic curve (AUROC) of 0.721 and a Macro F1 score of 0.616. The UCSF BERT-MS model, capable of leveraging multiple pieces of notes, surpassed the UCSF-BERT model in both AUROC and Macro-F1. Our feature importance analysis identified several clinically intuitive social determinants of health that potentially contribute to disparities in treatment.DiscussionLeveraging social work notes can be instrumental in identifying disparities in clinical decision-making. Hypotheses generated in an automated way could be used to guide patient-specific quality improvement interventions. Further validation with diverse clinical outcomes and prospective studies is essential.ConclusionsOur findings indicate that significant disparities exist among breast cancer patients receiving different types of therapies based on social determinants of health. Social work reports play a crucial role in understanding these disparities in clinical decision-making.

The release of GPT-4 and other large language models (LLMs) has the potential to transform healthcare. However, existing research evaluating LLM performance on real-world clinical notes is limited. Here, we conduct a highly-powered study to determine whether LLMs can provide clinical recommendations for three tasks (admission status, radiological investigation(s) request status, and antibiotic prescription status) using clinical notes from the Emergency Department. We randomly selected 10,000 Emergency Department visits to evaluate the accuracy of zero-shot, GPT-3.5-turbo- and GPT-4-turbo-generated clinical recommendations across four different prompting strategies. We found that both GPT-4-turbo and GPT-3.5-turbo performed poorly compared to a resident physician, with accuracy scores 8% and 24%, respectively, lower than physician on average. Both LLMs tended to be overly cautious in its recommendations, with high sensitivity at the cost of specificity. Our findings demonstrate that, while early evaluations of the clinical use of LLMs are promising, LLM performance must be significantly improved before their deployment as decision support systems for clinical recommendations and other complex tasks.

ObjectiveAlthough supervised machine learning is popular for information extraction from clinical notes, creating large annotated datasets requires extensive domain expertise and is time-consuming. Meanwhile, large language models (LLMs) have demonstrated promising transfer learning capability. In this study, we explored whether recent LLMs could reduce the need for large-scale data annotations.Materials and methodsWe curated a dataset of 769 breast cancer pathology reports, manually labeled with 12 categories, to compare zero-shot classification capability of the following LLMs: GPT-4, GPT-3.5, Starling, and ClinicalCamel, with task-specific supervised classification performance of 3 models: random forests, long short-term memory networks with attention (LSTM-Att), and the UCSF-BERT model.ResultsAcross all 12 tasks, the GPT-4 model performed either significantly better than or as well as the best supervised model, LSTM-Att (average macro F1-score of 0.86 vs 0.75), with advantage on tasks with high label imbalance. Other LLMs demonstrated poor performance. Frequent GPT-4 error categories included incorrect inferences from multiple samples and from history, and complex task design, and several LSTM-Att errors were related to poor generalization to the test set.DiscussionOn tasks where large annotated datasets cannot be easily collected, LLMs can reduce the burden of data labeling. However, if the use of LLMs is prohibitive, the use of simpler models with large annotated datasets can provide comparable results.ConclusionsGPT-4 demonstrated the potential to speed up the execution of clinical NLP studies by reducing the need for large annotated datasets. This may increase the utilization of NLP-based variables and outcomes in clinical studies.

BackgroundDry eye disease (DED) is a complex disease of the ocular surface, and its associated factors are important for understanding and effectively treating DED. ObjectiveThis study aimed to provide an integrative and personalized model of DED by making an explanatory model of DED using as many factors as possible from the Korea National Health and Nutrition Examination Survey (KNHANES) data. MethodsUsing KNHANES data for 2012 (4391 sample cases), a point-based scoring system was created for ranking factors associated with DED and assessing patient-specific DED risk. First, decision trees and lasso were used to classify continuous factors and to select important factors, respectively. Next, a survey-weighted multiple logistic regression was trained using these factors, and points were assigned using the regression coefficients. Finally, network graphs of partial correlations between factors were utilized to study the interrelatedness of DED-associated factors. ResultsThe point-based model achieved an area under the curve of 0.70 (95% CI 0.61-0.78), and 13 of 78 factors considered were chosen. Important factors included sex (+9 points for women), corneal refractive surgery (+9 points), current depression (+7 points), cataract surgery (+7 points), stress (+6 points), age (54-66 years; +4 points), rhinitis (+4 points), lipid-lowering medication (+4 points), and intake of omega-3 (0.43%-0.65% kcal/day; −4 points). Among these, the age group 54 to 66 years had high centrality in the network, whereas omega-3 had low centrality. ConclusionsIntegrative understanding of DED was possible using the machine learning–based model and network-based factor analysis. This method for finding important risk factors and identifying patient-specific risk could be applied to other multifactorial diseases.

ObjectivesVedolizumab (VDZ) and ustekinumab (UST) are second-line treatments in pediatric patients with ulcerative colitis (UC) refractory to antitumor necrosis factor (anti-TNF) therapy. Pediatric studies comparing the effectiveness of these medications are lacking. Using a registry from ImproveCareNow (ICN), a global research network in pediatric inflammatory bowel disease, we compared the effectiveness of UST and VDZ in anti-TNF refractory UC.MethodsWe performed a propensity-score weighted regression analysis to compare corticosteroid-free clinical remission (CFCR) at 6 months from starting second-line therapy. Sensitivity analyses tested the robustness of our findings to different ways of handling missing outcome data. Secondary analyses evaluated alternative proxies of response and infection risk.ResultsOur cohort included 262 patients on VDZ and 74 patients on UST. At baseline, the two groups differed on their mean pediatric UC activity index (PUCAI) (p = 0.03) but were otherwise similar. At Month 6, 28.3% of patients on VDZ and 25.8% of those on UST achieved CFCR (p = 0.76). Our primary model showed no difference in CFCR (odds ratio: 0.81; 95% confidence interval [CI]: 0.41-1.59) (p = 0.54). The time to biologic discontinuation was similar in both groups (hazard ratio: 1.26; 95% CI: 0.76-2.08) (p = 0.36), with the reference group being VDZ, and we found no differences in clinical response, growth parameters, hospitalizations, surgeries, infections, or malignancy risk. Sensitivity analyses supported these findings of similar effectiveness.ConclusionsUST and VDZ are similarly effective for inducing clinical remission in anti-TNF refractory UC in pediatric patients. Providers should consider safety, tolerability, cost, and comorbidities when deciding between these therapies.

Digital therapeutics (DTx) are a somewhat novel class of US Food and Drug Administration-regulated software that help patients prevent, manage, or treat disease. Here, we use natural language processing to characterise registered DTx clinical trials and provide insights into the clinical development landscape for these novel therapeutics. We identified 449 DTx clinical trials, initiated or expected to be initiated between 2010 and 2030, from ClinicalTrials.gov using 27 search terms, and available data were analysed, including trial durations, locations, MeSH categories, enrolment, and sponsor types. Topic modelling of eligibility criteria, done with BERTopic, showed that DTx trials frequently exclude patients on the basis of age, comorbidities, pregnancy, language barriers, and digital determinants of health, including smartphone or data plan access. Our comprehensive overview of the DTx development landscape highlights challenges in designing inclusive DTx clinical trials and presents opportunities for clinicians and researchers to address these challenges. Finally, we provide an interactive dashboard for readers to conduct their own analyses.

ObjectiveExisting research on social determinants of health (SDoH) predominantly focuses on physician notes and structured data within electronic medical records. This study posits that social work notes are an untapped, potentially rich source for SDoH information. We hypothesize that clinical notes recorded by social workers, whose role is to ameliorate social and economic factors, might provide a complementary information source of data on SDoH compared to physician notes, which primarily concentrate on medical diagnoses and treatments. We aimed to use word frequency analysis and topic modeling to identify prevalent terms and robust topics of discussion within a large cohort of social work notes including both outpatient and in-patient consultations.Materials and methodsWe retrieved a diverse, deidentified corpus of 0.95 million clinical social work notes from 181 644 patients at the University of California, San Francisco. We conducted word frequency analysis related to ICD-10 chapters to identify prevalent terms within the notes. We then applied Latent Dirichlet Allocation (LDA) topic modeling analysis to characterize this corpus and identify potential topics of discussion, which was further stratified by note types and disease groups.ResultsWord frequency analysis primarily identified medical-related terms associated with specific ICD10 chapters, though it also detected some subtle SDoH terms. In contrast, the LDA topic modeling analysis extracted 11 topics explicitly related to social determinants of health risk factors, such as financial status, abuse history, social support, risk of death, and mental health. The topic modeling approach effectively demonstrated variations between different types of social work notes and across patients with different types of diseases or conditions.DiscussionOur findings highlight LDA topic modeling's effectiveness in extracting SDoH-related themes and capturing variations in social work notes, demonstrating its potential for informing targeted interventions for at-risk populations.ConclusionSocial work notes offer a wealth of unique and valuable information on an individual's SDoH. These notes present consistent and meaningful topics of discussion that can be effectively analyzed and utilized to improve patient care and inform targeted interventions for at-risk populations.

Rheumatoid arthritis (RA) management leans toward achieving remission or low disease activity. In this study, we conducted single-cell RNA sequencing (scRNA-Seq) of peripheral blood mononuclear cells (PBMCs) from 36 individuals (18 patients with RA and 18 matched controls, accounting for age, sex, race, and ethnicity), to identify disease-relevant cell subsets and cell type-specific signatures associated with disease activity. Our analysis revealed 18 distinct PBMC subsets, including an IFN-induced transmembrane 3-overexpressing (IFITM3-overexpressing) IFN-activated monocyte subset. We observed an increase in CD4+ T effector memory cells in patients with moderate-high disease activity (DAS28-CRP ≥ 3.2) and a decrease in nonclassical monocytes in patients with low disease activity or remission (DAS28-CRP < 3.2). Pseudobulk analysis by cell type identified 168 differentially expressed genes between RA and matched controls, with a downregulation of proinflammatory genes in the γδ T cell subset, alteration of genes associated with RA predisposition in the IFN-activated subset, and nonclassical monocytes. Additionally, we identified a gene signature associated with moderate-high disease activity, characterized by upregulation of proinflammatory genes such as TNF, JUN, EGR1, IFIT2, MAFB, and G0S2 and downregulation of genes including HLA-DQB1, HLA-DRB5, and TNFSF13B. Notably, cell-cell communication analysis revealed an upregulation of signaling pathways, including VISTA, in both moderate-high and remission-low disease activity contexts. Our findings provide valuable insights into the systemic cellular and molecular mechanisms underlying RA disease activity.

Detecting epistatic drivers of human phenotypes is a considerable challenge. Traditional approaches use regression to sequentially test multiplicative interaction terms involving pairs of genetic variants. For higher-order interactions and genome-wide large-scale data, this strategy is computationally intractable. Moreover, multiplicative terms used in regression modeling may not capture the form of biological interactions. Building on the Predictability, Computability, Stability (PCS) framework, we introduce the epiTree pipeline to extract higher-order interactions from genomic data using tree-based models. The epiTree pipeline first selects a set of variants derived from tissue-specific estimates of gene expression. Next, it uses iterative random forests (iRF) to search training data for candidate Boolean interactions (pairwise and higher-order). We derive significance tests for interactions, based on a stabilized likelihood ratio test, by simulating Boolean tree-structured null (no epistasis) and alternative (epistasis) distributions on hold-out test data. Finally, our pipeline computes PCS epistasis p-values that probabilisticly quantify improvement in prediction accuracy via bootstrap sampling on the test set. We validate the epiTree pipeline in two case studies using data from the UK Biobank: predicting red hair and multiple sclerosis (MS). In the case of predicting red hair, epiTree recovers known epistatic interactions surrounding MC1R and novel interactions, representing non-linearities not captured by logistic regression models. In the case of predicting MS, a more complex phenotype than red hair, epiTree rankings prioritize novel interactions surrounding HLA-DRB1, a variant previously associated with MS in several populations. Taken together, these results highlight the potential for epiTree rankings to help reduce the design space for follow up experiments.

Recent developments in large language models (LLMs) have unlocked opportunities for healthcare, from information synthesis to clinical decision support. These LLMs are not just capable of modeling language, but can also act as intelligent “agents” that interact with stakeholders in open-ended conversations and even influence clinical decision-making. Rather than relying on benchmarks that measure a model’s ability to process clinical data or answer standardized test questions, LLM agents can be modeled in high-fidelity simulations of clinical settings and should be assessed for their impact on clinical workflows. These evaluation frameworks, which we refer to as “Artificial Intelligence Structured Clinical Examinations” (“AI-SCE”), can draw from comparable technologies where machines operate with varying degrees of self-governance, such as self-driving cars, in dynamic environments with multiple stakeholders. Developing these robust, real-world clinical evaluations will be crucial towards deploying LLM agents in medical settings.

Acute kidney injury (AKI) is a major postoperative complication that lacks established intraoperative predictors. Our objective was to develop a prediction model using preoperative and high-frequency intraoperative data for postoperative AKI. In this retrospective cohort study, we evaluated 77,428 operative cases at a single academic center between 2016 and 2022. A total of 11,212 cases with serum creatinine (sCr) data were included in the analysis. Then, 8519 cases were randomly assigned to the training set and the remainder to the validation set. Fourteen preoperative and twenty intraoperative variables were evaluated using elastic net followed by hierarchical group least absolute shrinkage and selection operator (LASSO) regression. The training set was 56% male and had a median [IQR] age of 62 (51-72) and a 6% AKI rate. Retained model variables were preoperative sCr values, the number of minutes meeting cutoffs for urine output, heart rate, perfusion index intraoperatively, and the total estimated blood loss. The area under the receiver operator characteristic curve was 0.81 (95% CI, 0.77-0.85). At a score threshold of 0.767, specificity was 77% and sensitivity was 74%. A web application that calculates the model score is available online. Our findings demonstrate the utility of intraoperative time series data for prediction problems, including a new potential use of the perfusion index. Further research is needed to evaluate the model in clinical settings.

Sympathetic activation during cold exposure increases adipocyte thermogenesis via the expression of mitochondrial protein uncoupling protein 1 (UCP1)1. The propensity of adipocytes to express UCP1 is under a critical influence of the adipose microenvironment and varies between sexes and among various fat depots2-7. Here we report that mammary gland ductal epithelial cells in the adipose niche regulate cold-induced adipocyte UCP1 expression in female mouse subcutaneous white adipose tissue (scWAT). Single-cell RNA sequencing shows that glandular luminal epithelium subtypes express transcripts that encode secretory factors controlling adipocyte UCP1 expression under cold conditions. We term these luminal epithelium secretory factors 'mammokines'. Using 3D visualization of whole-tissue immunofluorescence, we reveal sympathetic nerve-ductal contact points. We show that mammary ducts activated by sympathetic nerves limit adipocyte UCP1 expression via the mammokine lipocalin 2. In vivo and ex vivo ablation of mammary duct epithelium enhance the cold-induced adipocyte thermogenic gene programme in scWAT. Since the mammary duct network extends throughout most of the scWAT in female mice, females show markedly less scWAT UCP1 expression, fat oxidation, energy expenditure and subcutaneous fat mass loss compared with male mice, implicating sex-specific roles of mammokines in adipose thermogenesis. These results reveal a role of sympathetic nerve-activated glandular epithelium in adipocyte UCP1 expression and suggest that mammary duct luminal epithelium has an important role in controlling glandular adiposity.

ObjectivesClinical notes are a veritable treasure trove of information on a patient's disease progression, medical history, and treatment plans, yet are locked in secured databases accessible for research only after extensive ethics review. Removing personally identifying and protected health information (PII/PHI) from the records can reduce the need for additional Institutional Review Boards (IRB) reviews. In this project, our goals were to: (1) develop a robust and scalable clinical text de-identification pipeline that is compliant with the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule for de-identification standards and (2) share routinely updated de-identified clinical notes with researchers.Materials and methodsBuilding on our open-source de-identification software called Philter, we added features to: (1) make the algorithm and the de-identified data HIPAA compliant, which also implies type 2 error-free redaction, as certified via external audit; (2) reduce over-redaction errors; and (3) normalize and shift date PHI. We also established a streamlined de-identification pipeline using MongoDB to automatically extract clinical notes and provide truly de-identified notes to researchers with periodic monthly refreshes at our institution.ResultsTo the best of our knowledge, the Philter V1.0 pipeline is currently the first and only certified, de-identified redaction pipeline that makes clinical notes available to researchers for nonhuman subjects' research, without further IRB approval needed. To date, we have made over 130 million certified de-identified clinical notes available to over 600 UCSF researchers. These notes were collected over the past 40 years, and represent data from 2757016 UCSF patients.

ObjectivesAs the real-world electronic health record (EHR) data continue to grow exponentially, novel methodologies involving artificial intelligence (AI) are becoming increasingly applied to enable efficient data-driven learning and, ultimately, to advance healthcare. Our objective is to provide readers with an understanding of evolving computational methods and help in deciding on methods to pursue.Target audienceThe sheer diversity of existing methods presents a challenge for health scientists who are beginning to apply computational methods to their research. Therefore, this tutorial is aimed at scientists working with EHR data who are early entrants into the field of applying AI methodologies.ScopeThis manuscript describes the diverse and growing AI research approaches in healthcare data science and categorizes them into 2 distinct paradigms, the bottom-up and top-down paradigms to provide health scientists venturing into artificial intelligent research with an understanding of the evolving computational methods and help in deciding on methods to pursue through the lens of real-world healthcare data.

Differential host responses in coronavirus disease 2019 (COVID-19) and multisystem inflammatory syndrome in children (MIS-C) remain poorly characterized. Here, we use next-generation sequencing to longitudinally analyze blood samples from pediatric patients with COVID-19 or MIS-C across three hospitals. Profiling of plasma cell-free nucleic acids uncovers distinct signatures of cell injury and death between COVID-19 and MIS-C, with increased multiorgan involvement in MIS-C encompassing diverse cell types, including endothelial and neuronal cells, and an enrichment of pyroptosis-related genes. Whole-blood RNA profiling reveals upregulation of similar pro-inflammatory pathways in COVID-19 and MIS-C but also MIS-C-specific downregulation of T cell-associated pathways. Profiling of plasma cell-free RNA and whole-blood RNA in paired samples yields different but complementary signatures for each disease state. Our work provides a systems-level view of immune responses and tissue damage in COVID-19 and MIS-C and informs future development of new disease biomarkers.

BackgroundAdverse events of special interest (AESIs) were pre-specified to be monitored for the COVID-19 vaccines. Some AESIs are not only associated with the vaccines, but with COVID-19. Our aim was to characterise the incidence rates of AESIs following SARS-CoV-2 infection in patients and compare these to historical rates in the general population.MethodsA multi-national cohort study with data from primary care, electronic health records, and insurance claims mapped to a common data model. This study's evidence was collected between Jan 1, 2017 and the conclusion of each database (which ranged from Jul 2020 to May 2022). The 16 pre-specified prevalent AESIs were: acute myocardial infarction, anaphylaxis, appendicitis, Bell's palsy, deep vein thrombosis, disseminated intravascular coagulation, encephalomyelitis, Guillain- Barré syndrome, haemorrhagic stroke, non-haemorrhagic stroke, immune thrombocytopenia, myocarditis/pericarditis, narcolepsy, pulmonary embolism, transverse myelitis, and thrombosis with thrombocytopenia. Age-sex standardised incidence rate ratios (SIR) were estimated to compare post-COVID-19 to pre-pandemic rates in each of the databases.FindingsSubstantial heterogeneity by age was seen for AESI rates, with some clearly increasing with age but others following the opposite trend. Similarly, differences were also observed across databases for same health outcome and age-sex strata. All studied AESIs appeared consistently more common in the post-COVID-19 compared to the historical cohorts, with related meta-analytic SIRs ranging from 1.32 (1.05 to 1.66) for narcolepsy to 11.70 (10.10 to 13.70) for pulmonary embolism.InterpretationOur findings suggest all AESIs are more common after COVID-19 than in the general population. Thromboembolic events were particularly common, and over 10-fold more so. More research is needed to contextualise post-COVID-19 complications in the longer term.FundingNone.

The incidence and biochemical consequences of rare tumor subtypes are often hard to study. Fibrolamellar liver cancer (FLC) is a rare malignancy affecting adolescents and young adults. To better characterize the incidence and biochemical consequences of this disease, we combined a comprehensive analysis of the electronic medical record and national payer data and found that FLC incidence is likely five to eight times higher than previous estimates. By employing unsupervised learning on clinical laboratory data from patients with hyperammonemia, we find that FLC-associated hyperammonemia mirrors metabolic dysregulation in urea cycle disorders. Our findings demonstrate that advanced computational analysis of rich clinical datasets can provide key clinical and biochemical insights into rare cancers.

Study designA retrospective study at a single academic institution.ObjectiveThe purpose of this study is to utilize machine learning to predict hospital length of stay (LOS) and discharge disposition following adult elective spine surgery, and to compare performance metrics of machine learning models to the American College of Surgeon's National Surgical Quality Improvement Program's (ACS NSQIP) prediction calculator.Summary of background dataA total of 3678 adult patients undergoing elective spine surgery between 2014 and 2019, acquired from the electronic health record.MethodsPatients were divided into three stratified cohorts: cervical degenerative, lumbar degenerative, and adult spinal deformity groups. Predictive variables included demographics, body mass index, surgical region, surgical invasiveness, surgical approach, and comorbidities. Regression, classification trees, and least absolute shrinkage and selection operator (LASSO) were used to build predictive models. Validation of the models was conducted on 16% of patients (N=587), using area under the receiver operator curve (AUROC), sensitivity, specificity, and correlation. Patient data were manually entered into the ACS NSQIP online risk calculator to compare performance. Outcome variables were discharge disposition (home vs. rehabilitation) and LOS (days).ResultsOf 3678 patients analyzed, 51.4% were male (n=1890) and 48.6% were female (n=1788). The average LOS was 3.66 days. In all, 78% were discharged home and 22% discharged to rehabilitation. Compared with NSQIP (Pearson R2 =0.16), the predictions of poisson regression ( R2 =0.29) and LASSO ( R2 =0.29) models were significantly more correlated with observed LOS ( P =0.025 and 0.004, respectively). Of the models generated to predict discharge location, logistic regression yielded an AUROC of 0.79, which was statistically equivalent to the AUROC of 0.75 for NSQIP ( P =0.135).ConclusionThe predictive models developed in this study can enable accurate preoperative estimation of LOS and risk of rehabilitation discharge for adult patients undergoing elective spine surgery. The demonstrated models exhibited better performance than NSQIP for prediction of LOS and equivalent performance to NSQIP for prediction of discharge location.

BackgroundThe advent of clinical trial data sharing platforms has created opportunities for making new discoveries and answering important questions using already collected data. However, existing methods for meta-analyzing these data require the presence of shared control groups across studies, significantly limiting the number of questions that can be confidently addressed. We sought to develop a method for meta-analyzing potentially heterogeneous clinical trials even in the absence of a common control group.MethodsThis work was conducted within the context of a broader effort to study comparative efficacy in Crohn's disease. Following a search of clnicaltrials.gov we obtained access to the individual participant data from nine trials of FDA-approved treatments in Crohn's Disease (N = 3392). We developed a method involving sequences of regression and simulation to separately model the placebo- and drug-attributable effects, and to simulate head-to-head trials against an appropriately normalized background. We validated this method by comparing the outcome of a simulated trial comparing the efficacies of adalimumab and ustekinumab against the recently published results of SEAVUE, an actual head-to-head trial of these drugs. This study was pre-registered on PROSPERO (#157,827) prior to the completion of SEAVUE.ResultsUsing our method of sequential regression and simulation, we compared the week eight outcomes of two virtual cohorts subject to the same patient selection criteria as SEAVUE and treated with adalimumab or ustekinumab. Our primary analysis replicated the corresponding published results from SEAVUE (p = 0.9). This finding proved stable under multiple sensitivity analyses.ConclusionsThis new method may help reduce the bias of individual participant data meta-analyses, expand the scope of what can be learned from these already-collected data, and reduce the costs of obtaining high-quality evidence to guide patient care.

Digital health technologies have been in use for many years in a wide spectrum of healthcare scenarios. This narrative review outlines the current use and the future strategies and significance of digital health technologies in modern healthcare applications. It covers the current state of the scientific field (delineating major strengths, limitations, and applications) and envisions the future impact of relevant emerging key technologies. Furthermore, we attempt to provide recommendations for innovative approaches that would accelerate and benefit the research, translation and utilization of digital health technologies.

There is an increased interest in utilizing real-world data (RWD) for pharmaceutical research and regulatory decision-making. The development and use of pediatric medicines could benefit greatly from real-world data studies given nearly half of drugs prescribed to children are "off-label", meaning there is a lack of pediatric-specific evidence from controlled trials, while there is an abundance of data from routine clinical practice. Currently, the use of real-world data, such as data from electronic health records, is lacking in pediatric research, especially within infectious diseases. Here, we discuss opportunities and challenges for real-world data to generate evidence on the optimal treatment and management of infectious diseases in children.

Chronic low back pain (cLBP) afflicts 8. 2% of adults in the United States, and is the leading global cause of disability. Neuropsychiatric co-morbidities including anxiety, depression, and substance abuse- are common in cLBP patients. In particular, cLBP is a risk factor for opioid addiction, as more than 50% of opioid prescriptions in the United States are for cLBP. Misuse of these prescriptions is a common precursor to addiction. While associations between cLBP and neuropsychiatric disorders are well established, causal relationships for the most part are unknown. Developing effective treatments for cLBP, and associated co-morbidities, requires identifying and understanding causal relationships. Rigorous methods for causal inference, a process for quantifying causal effects from observational data, have been developed over the past 30 years. In this review we first discuss the conceptual model of cLBP that current treatments are based on, and how gaps in causal knowledge contribute to poor clinical outcomes. We then present cLBP as a "Big Data" problem and identify how advanced analytic techniques may close knowledge gaps and improve clinical outcomes. We will focus on causal discovery, which is a data-driven method that uses artificial intelligence (AI) and high dimensional datasets to identify causal structures, discussing both constraint-based (PC and Fast Causal Inference) and score-based (Fast Greedy Equivalent Search) algorithms.

BackgroundRandomized trials are the gold-standard for clinical evidence generation, but they can sometimes be limited by infeasibility and unclear generalizability to real-world practice. External control arm (ECA) studies may help address this evidence gaps by constructing retrospective cohorts that closely emulate prospective ones. Experience in constructing these outside the context of rare diseases or cancer is limited. We piloted an approach for developing an ECA in Crohn's disease using electronic health records (EHR) data.MethodsWe queried EHR databases and manually screened records at the University of California, San Francisco to identify patients meeting the eligibility criteria of TRIDENT, a recently completed interventional trial involving an ustekinumab reference arm. We defined timepoints to balance missing data and bias. We compared imputation models by their impacts on cohort membership and outcomes. We assessed the accuracy of algorithmic data curation against manual review. Lastly, we assessed disease activity following treatment with ustekinumab.ResultsScreening identified 183 patients. 30% of the cohort had missing baseline data. Nonetheless, cohort membership and outcomes were robust to the method of imputation. Algorithms for ascertaining non-symptom-based elements of disease activity using structured data were accurate against manual review. The cohort consisted of 56 patients, exceeding planned enrollment in TRIDENT. 34% of the cohort was in steroid-free remission at week 24.ConclusionWe piloted an approach for creating an ECA in Crohn's disease from EHR data by using a combination of informatics and manual methods. However, our study reveals significant missing data when standard-of-care clinical data are repurposed. More work will be needed to improve the alignment of trial design with typical patterns of clinical practice, and thereby enable a future of more robust ECAs in chronic diseases like Crohn's disease.

Background As diagnostic tests for COVID-19 were broadly deployed under Emergency Use Authorization, there emerged a need to understand the real-world utilization and performance of serological testing across the United States. Methods Six health systems contributed electronic health records and/or claims data, jointly developed a master protocol, and used it to execute the analysis in parallel. We used descriptive statistics to examine demographic, clinical, and geographic characteristics of serology testing among patients with RNA positive for SARS-CoV-2. Results Across datasets, we observed 930,669 individuals with positive RNA for SARS-CoV-2. Of these, 35,806 (4%) were serotested within 90 days; 15% of which occurred 30%) in some datasets—limiting our ability to examine differences in serological testing by race. In datasets where race/ethnicity information was available, we observed a greater distribution of White individuals among those serotested; however, the time between RNA and serology tests appeared shorter in Black compared to White individuals. Test manufacturer data was available in half of the datasets contributing to the analysis. Conclusion Our results inform the underlying context of serotesting during the first year of the COVID-19 pandemic and differences observed between claims and EHR data sources–a critical first step to understanding the real-world accuracy of serological tests. Incomplete reporting of race/ethnicity data and a limited ability to link test manufacturer data, lab results, and clinical data challenge the ability to assess the real-world performance of SARS-CoV-2 tests in different contexts and the overall U.S. response to current and future disease pandemics.

Background Real-world performance of COVID-19 diagnostic tests under Emergency Use Authorization (EUA) must be assessed. We describe overall trends in the performance of serology tests in the context of real-world implementation. Methods Six health systems estimated the odds of seropositivity and positive percent agreement (PPA) of serology test among people with confirmed SARS-CoV-2 infection by molecular test. In each dataset, we present the odds ratio and PPA, overall and by key clinical, demographic, and practice parameters. Results A total of 15,615 people were observed to have at least one serology test 14–90 days after a positive molecular test for SARS-CoV-2. We observed higher PPA in Hispanic (PPA range: 79–96%) compared to non-Hispanic (60–89%) patients; in those presenting with at least one COVID-19 related symptom (69–93%) as compared to no such symptoms (63–91%); and in inpatient (70–97%) and emergency department (93–99%) compared to outpatient (63–92%) settings across datasets. PPA was highest in those with diabetes (75–94%) and kidney disease (83–95%); and lowest in those with auto-immune conditions or who are immunocompromised (56–93%). The odds ratios (OR) for seropositivity were higher in Hispanics compared to non-Hispanics (OR range: 2.59–3.86), patients with diabetes (1.49–1.56), and obesity (1.63–2.23); and lower in those with immunocompromised or autoimmune conditions (0.25–0.70), as compared to those without those comorbidities. In a subset of three datasets with robust information on serology test name, seven tests were used, two of which were used in multiple settings and met the EUA requirement of PPA ≥87%. Tests performed similarly across datasets. Conclusion Although the EUA requirement was not consistently met, more investigation is needed to understand how serology and molecular tests are used, including indication and protocol fidelity. Improved data interoperability of test and clinical/demographic data are needed to enable rapid assessment of the real-world performance of in vitro diagnostic tests.

Biomedicine is undergoing a revolution driven by high throughput and connective computing that is transforming medical research and practice. Using oncology as an example, the speed and capacity of genomic sequencing technologies is advancing the utility of individual genetic profiles for anticipating risk and targeting therapeutics. The goal is to enable an era of “P4” medicine that will become increasingly more predictive, personalized, preemptive, and participative over time. This vision hinges on leveraging potentially innovative and disruptive technologies in medicine to accelerate discovery and to reorient clinical practice for patient-centered care. Based on a panel discussion at the Medicine 2.0 conference in Boston with representatives from the National Cancer Institute, Moffitt Cancer Center, and Stanford University School of Medicine, this paper explores how emerging sociotechnical frameworks, informatics platforms, and health-related policy can be used to encourage data liquidity and innovation. This builds on the Institute of Medicine’s vision for a “rapid learning health care system” to enable an open source, population-based approach to cancer prevention and control.

Pathologists can label pathologies differently, making it challenging to yield consistent assessments in the absence of one ground truth. To address this problem, we present a deep learning (DL) approach that draws on a cohort of experts, weighs each contribution, and is robust to noisy labels. We collected 100,495 annotations on 20,099 candidate amyloid beta neuropathologies (cerebral amyloid angiopathy (CAA), and cored and diffuse plaques) from three institutions, independently annotated by five experts. DL methods trained on a consensus-of-two strategy yielded 12.6-26% improvements by area under the precision recall curve (AUPRC) when compared to those that learned individualized annotations. This strategy surpassed individual-expert models, even when unfairly assessed on benchmarks favoring them. Moreover, ensembling over individual models was robust to hidden random annotators. In blind prospective tests of 52,555 subsequent expert-annotated images, the models labeled pathologies like their human counterparts (consensus model AUPRC = 0.74 cored; 0.69 CAA). This study demonstrates a means to combine multiple ground truths into a common-ground DL model that yields consistent diagnoses informed by multiple and potentially variable expert opinions.

BackgroundAngiotensin-converting enzyme 2 (ACE2) is the cell-entry receptor for SARS-CoV-2. It plays critical roles in both the transmission and the pathogenesis of COVID-19. Comprehensive profiling of ACE2 expression patterns could reveal risk factors of severe COVID-19 illness. While the expression of ACE2 in healthy human tissues has been well characterized, it is not known which diseases and drugs might be associated with ACE2 expression.ResultsWe develop GENEVA (GENe Expression Variance Analysis), a semi-automated framework for exploring massive amounts of RNA-seq datasets. We apply GENEVA to 286,650 publicly available RNA-seq samples to identify any previously studied experimental conditions that could be directly or indirectly associated with ACE2 expression. We identify multiple drugs, genetic perturbations, and diseases that are associated with the expression of ACE2, including cardiomyopathy, HNF1A overexpression, and drug treatments with RAD140 and itraconazole. Our joint analysis of seven datasets confirms ACE2 upregulation in all cardiomyopathy categories. Using electronic health records data from 3936 COVID-19 patients, we demonstrate that patients with pre-existing cardiomyopathy have an increased mortality risk than age-matched patients with other cardiovascular conditions. GENEVA is applicable to any genes of interest and is freely accessible at http://genevatool.org .ConclusionsThis study identifies multiple diseases and drugs that are associated with the expression of ACE2. The effect of these conditions should be carefully studied in COVID-19 patients. In particular, our analysis identifies cardiomyopathy patients as a high-risk group, with increased ACE2 expression in the heart and increased mortality after SARS-COV-2 infection.

BackgroundAccurate, pragmatic risk stratification for postoperative delirium (POD) is necessary to target preventative resources toward high-risk patients. Machine learning (ML) offers a novel approach to leveraging electronic health record (EHR) data for POD prediction. We sought to develop and internally validate a ML-derived POD risk prediction model using preoperative risk features, and to compare its performance to models developed with traditional logistic regression.MethodsThis was a retrospective analysis of preoperative EHR data from 24,885 adults undergoing a procedure requiring anesthesia care, recovering in the main post-anesthesia care unit, and staying in the hospital at least overnight between December 2016 and December 2019 at either of two hospitals in a tertiary care health system. One hundred fifteen preoperative risk features including demographics, comorbidities, nursing assessments, surgery type, and other preoperative EHR data were used to predict postoperative delirium (POD), defined as any instance of Nursing Delirium Screening Scale ≥2 or positive Confusion Assessment Method for the Intensive Care Unit within the first 7 postoperative days. Two ML models (Neural Network and XGBoost), two traditional logistic regression models ("clinician-guided" and "ML hybrid"), and a previously described delirium risk stratification tool (AWOL-S) were evaluated using the area under the receiver operating characteristic curve (AUC-ROC), sensitivity, specificity, positive likelihood ratio, and positive predictive value. Model calibration was assessed with a calibration curve. Patients with no POD assessments charted or at least 20% of input variables missing were excluded.ResultsPOD incidence was 5.3%. The AUC-ROC for Neural Net was 0.841 [95% CI 0. 816-0.863] and for XGBoost was 0.851 [95% CI 0.827-0.874], which was significantly better than the clinician-guided (AUC-ROC 0.763 [0.734-0.793], p

Opal is the first published example of a full-stack platform infrastructure for an implementation science designed for ML in anesthesia that solves the problem of leveraging ML for clinical decision support. Users interact with a secure online Opal web application to select a desired operating room (OR) case cohort for data extraction, visualize datasets with built-in graphing techniques, and run in-client ML or extract data for external use. Opal was used to obtain data from 29,004 unique OR cases from a single academic institution for pre-operative prediction of post-operative acute kidney injury (AKI) based on creatinine KDIGO criteria using predictors which included pre-operative demographic, past medical history, medications, and flowsheet information. To demonstrate utility with unsupervised learning, Opal was also used to extract intra-operative flowsheet data from 2995 unique OR cases and patients were clustered using PCA analysis and k-means clustering. A gradient boosting machine model was developed using an 80/20 train to test ratio and yielded an area under the receiver operating curve (ROC-AUC) of 0.85 with 95% CI [0.80-0.90]. At the default probability decision threshold of 0.5, the model sensitivity was 0.9 and the specificity was 0.8. K-means clustering was performed to partition the cases into two clusters and for hypothesis generation of potential groups of outcomes related to intraoperative vitals. Opal's design has created streamlined ML functionality for researchers and clinicians in the perioperative setting and opens the door for many future clinical applications, including data mining, clinical simulation, high-frequency prediction, and quality improvement.

ObjectivePediatric focused assessment with sonography for trauma (FAST) is a sequence of ultrasound views rapidly performed by clinicians to diagnose hemorrhage. A technical limitation of FAST is the lack of expertise to consistently acquire all required views. We sought to develop an accurate deep learning view classifier using a large heterogeneous dataset of clinician-performed pediatric FAST.MethodsWe developed and conducted a retrospective cohort analysis of a deep learning view classifier on real-world FAST studies performed on injured children less than 18 years old in two pediatric emergency departments by 30 different clinicians. FAST was randomly distributed to training, validation, and test datasets, 70:20:10; each child was represented in only one dataset. The primary outcome was view classifier accuracy for video clips and still frames.ResultsThere were 699 FAST studies, representing 4925 video clips and 1,062,612 still frames, performed by 30 different clinicians. The overall classification accuracy was 97.8% (95% confidence interval [CI]: 96.0-99.0) for video clips and 93.4% (95% CI: 93.3-93.6) for still frames. Per view still frames were classified with an accuracy: 96.0% (95% CI: 95.9-96.1) cardiac, 99.8% (95% CI: 99.8-99.8) pleural, 95.2% (95% CI: 95.0-95.3) abdominal upper quadrants, and 95.9% (95% CI: 95.8-96.0) suprapubic.ConclusionA deep learning classifier can accurately predict pediatric FAST views. Accurate view classification is important for quality assurance and feasibility of a multi-stage deep learning FAST model to enhance the evaluation of injured children.

ObjectiveThe Pediatric Emergency Care Applied Research Network (PECARN) has developed a clinical-decision instrument (CDI) to identify children at very low risk of intra-abdominal injury. However, the CDI has not been externally validated. We sought to vet the PECARN CDI with the Predictability Computability Stability (PCS) data science framework, potentially increasing its chance of a successful external validation.Materials & methodsWe performed a secondary analysis of two prospectively collected datasets: PECARN (12,044 children from 20 emergency departments) and an independent external validation dataset from the Pediatric Surgical Research Collaborative (PedSRC; 2,188 children from 14 emergency departments). We used PCS to reanalyze the original PECARN CDI along with new interpretable PCS CDIs developed using the PECARN dataset. External validation was then measured on the PedSRC dataset.ResultsThree predictor variables (abdominal wall trauma, Glasgow Coma Scale Score

In microscopy-based drug screens, fluorescent markers carry critical information on how compounds affect different biological processes. However, practical considerations, such as the labor and preparation formats needed to produce different image channels, hinders the use of certain fluorescent markers. Consequently, completed screens may lack biologically informative but experimentally impractical markers. Here, we present a deep learning method for overcoming these limitations. We accurately generated predicted fluorescent signals from other related markers and validated this new machine learning (ML) method on two biologically distinct datasets. We used the ML method to improve the selection of biologically active compounds for Alzheimer's disease (AD) from a completed high-content high-throughput screen (HCS) that had only contained the original markers. The ML method identified novel compounds that effectively blocked tau aggregation, which had been missed by traditional screening approaches unguided by ML. The method improved triaging efficiency of compound rankings over conventional rankings by raw image channels. We reproduced this ML pipeline on a biologically independent cancer-based dataset, demonstrating its generalizability. The approach is disease-agnostic and applicable across diverse fluorescence microscopy datasets.