Search

Background: Previous studies in medical imaging have shown disparate abilities of artificial intelligence (AI) to detect a person's race, yet there is no known correlation for race on medical imaging that would be obvious to human experts when interpreting the images. We aimed to conduct a comprehensive evaluation of the ability of AI to recognise a patient's racial identity from medical images., Methods: Using private (Emory CXR, Emory Chest CT, Emory Cervical Spine, and Emory Mammogram) and public (MIMIC-CXR, CheXpert, National Lung Cancer Screening Trial, RSNA Pulmonary Embolism CT, and Digital Hand Atlas) datasets, we evaluated, first, performance quantification of deep learning models in detecting race from medical images, including the ability of these models to generalise to external environments and across multiple imaging modalities. Second, we assessed possible confounding of anatomic and phenotypic population features by assessing the ability of these hypothesised confounders to detect race in isolation using regression models, and by re-evaluating the deep learning models by testing them on datasets stratified by these hypothesised confounding variables. Last, by exploring the effect of image corruptions on model performance, we investigated the underlying mechanism by which AI models can recognise race., Findings: In our study, we show that standard AI deep learning models can be trained to predict race from medical images with high performance across multiple imaging modalities, which was sustained under external validation conditions (x-ray imaging [area under the receiver operating characteristics curve (AUC) range 0·91-0·99], CT chest imaging [0·87-0·96], and mammography [0·81]). We also showed that this detection is not due to proxies or imaging-related surrogate covariates for race (eg, performance of possible confounders: body-mass index [AUC 0·55], disease distribution [0·61], and breast density [0·61]). Finally, we provide evidence to show that the ability of AI deep learning models persisted over all anatomical regions and frequency spectrums of the images, suggesting the efforts to control this behaviour when it is undesirable will be challenging and demand further study., Interpretation: The results from our study emphasise that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging., Funding: National Institute of Biomedical Imaging and Bioengineering, MIDRC grant of National Institutes of Health, US National Science Foundation, National Library of Medicine of the National Institutes of Health, and Taiwan Ministry of Science and Technology., Competing Interests: Declaration of interests MG has received speaker fees for a Harvard Medical School executive education class. HT has received consulting fees from Sirona medical, Arterys, and Biodata consortium. HT also owns lightbox AI, which provides expert annotation of medical images for radiology AI. MPL has received consulting fees from Bayer, Microsoft, Phillips, and Nines. MPL also owns stocks in Nines, SegMed, and Centaur. LAC has received support to attend meetings from MISTI Global Seed Funds. ATP has received payment for expert testimony from NCMIC insurance company. ATP also has a pending institutional patent for comorbidity prediction from radiology images. All other authors declare no competing interests., (Copyright © 2022 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license. Published by Elsevier Ltd.. All rights reserved.)

Various deep learning‐based video frame interpolation methods have been proposed in the past few years, but how to generate high quality interpolated frames in videos with large motions, complex backgrounds and rich textures is still a challenging issue. To deal with this limitation, a frame interpolation method based on residual blocks and feature pyramids is proposed. U‐Net is the main architecture of our method, which can capture multi‐layer information, segment objects from the background and obtain parameters with motion information to guide frame interpolation. However, the upsampling and subsampled of U‐Net will lose important information. In order to acquire more detailed contextual information, shortcut connection is used in the encoder basic module. At the same time, feature pyramid network is employed to capture features at different scales of the decoder to improve the representation of inter‐frame spatial‐temporal features. The experimental results show that the proposed method outperform the baseline methods in both of objective and subjective evaluations on different datasets. In particular, the method has obvious advantages on datasets which contain complex background. [ABSTRACT FROM AUTHOR]

The coronavirus disease 2019 (COVID-19) pandemic has caused millions of deaths around the world and revealed the need for data-driven models of pandemic spread. Accurate pandemic caseload forecasting allows informed policy decisions on the adoption of non-pharmaceutical interventions (NPIs) to reduce disease transmission. Using COVID-19 as an example, we present Pandemic conditional Ordinary Differential Equation (PAN-cODE), a deep learning method to forecast daily increases in pandemic infections and deaths. By using a deep conditional latent variable model, PAN-cODE can generate alternative caseload trajectories based on alternate adoptions of NPIs, allowing stakeholders to make policy decisions in an informed manner. PAN-cODE also allows caseload estimation for regions that are unseen during model training. We demonstrate that, despite using less detailed data and having fully automated training, PAN-cODE's performance is comparable to state-of-the-art methods on 4-week-ahead and 6-week-ahead forecasting. Finally, we highlight the ability of PAN-cODE to generate realistic alternative outcome trajectories on select US regions. [ABSTRACT FROM AUTHOR]

Personalized cancer treatments based on the molecular profile of a patient's tumor are an emerging and exciting class of treatments in oncology. As genomic tumor profiling is becoming more common, targeted treatments for specific molecular alterations are gaining traction. To discover new potential therapeutics that may apply to broad classes of tumors matching some molecular pattern, experimentalists and pharmacologists rely on high-throughput, in vitro screens of many compounds against many different cell lines. We propose a hierarchical Bayesian model of how cancer cell lines respond to drugs in these experiments and develop a method for fitting the model to real-world high-throughput screening data. Through a case study, the model is shown to capture nontrivial associations between molecular features and drug response, such as requiring both wild type TP53 and overexpression of MDM2 to be sensitive to Nutlin-3(a). In quantitative benchmarks, the model outperforms a standard approach in biology, with $\approx20\%$ lower predictive error on held out data. When combined with a conditional randomization testing procedure, the model discovers markers of therapeutic response that recapitulate known biology and suggest new avenues for investigation. All code for the article is publicly available at https://github.com/tansey/deep-dose-response. [ABSTRACT FROM AUTHOR]

Ex-ante online risk assessment for building emergency evacuation is essential to protect human life and property. Current risk assessment methods are limited by the tradeoff between accuracy and efficiency. In this paper, we propose an online method that overcomes this tradeoff based on multimedia data (e.g. videos data from surveillance cameras) and deep learning. The method consists of two parts. The first estimates the evacuee position as input for training the assessment model to then perform risk assessment in real scenarios. The second considers a social force model based on the evacuation simulation for the output of training model. We verify the proposed method in simulation and real scenarios. Model sensitivity analyses and large-scale tests demonstrate the usability and superiority of the proposed method. By the method, the computation time of risk assessment could be decreased from 10 minutes (by traditional simulation method) to 2.18 s. [ABSTRACT FROM AUTHOR]

Leak fault significantly affects the reliable and sustainable green hydrogen energy supply by renewable Power-to-Hydrogen (P2H2) system. Deep learning has been widely applied to automated leak fault detection and diagnosis of hydrogen systems. However, due to the intermittency of wind/solar-power-generation, existing approaches developed by monitored signals have the bad generalization to leak scenario under large volatility of power-to‑hydrogen production. This study introduces graph deep probability learning approach, in which an attention-based graph neural network (GNN) learns the dependency between installed sensors. Variational inference is integrated to model posterior distribution of sensor dependencies, by using which the leak fault under large volatility of power-to‑hydrogen production is detected and localized by using normal time-series signals under different timeslots of one day as training data. Experiment of hydrogen leak faults from P2H2 system is conducted to verify the more accuracy of proposed approach compared to 6 state-of-the-art approaches. Results demonstrated our proposed approach detects the leak fault more accurately with a higher AUC of 0.96 and successfully localizes all the leak faults. This study supports more reliable and efficient safety monitoring for upcoming renewable P2H2 system in future. • A graph deep probability learning approach is proposed to accurately detect and localize leak fault of P2H2 production. • Experiment of hydrogen leak is conducted to compare our proposed approach with 6 existing approaches. • Our approach achieves higher detection accuracy and localizes all faults when large volatility of P2H2 production exists. • Our proposed approach has better generalization to faults detection and diagnosis of upcoming renewable P2H2 system. [ABSTRACT FROM AUTHOR]

Online ride-hailing as an innovative travel mode becomes increasingly popular in cities around the world. To improve the efficiency of dispatching system of ride-hailing, many ride-hailing demand prediction models based on deep learning architecture have been proposed to reduce the gross travel distance between drivers and passengers. However, in most of these prediction models, only the error metrics are used for performance evaluation. There is scarce evidence on how much the demand prediction model reduces the unnecessary travel distance in reality. In this study, a multi-scenario-based method is proposed to evaluate technical potentials of the ride-hailing demand prediction model in ride-hailing dispatching system simulation. The ride-hailing dispatching is simulated in three scenarios: traditional dispatching system, prediction model-based dispatching system, and perfect prediction model-based dispatching system. One-month data of Didi Express service provided by Didi Chuxing GAIA Initiative in Chengdu is employed to support the simulation. Two terms, empty distance and relative performance, are introduced as the criteria of prediction model performance measurement. Simulation results reveal that the total empty distance reduced 1,164 km per day by using the prediction model compared with the traditional dispatching system. The relative performance is only 58.9% compared with the perfect prediction model-based dispatching system. [ABSTRACT FROM AUTHOR]

Significance: Automatic and accurate classification of three-dimensional (3-D) retinal optical coherence tomography (OCT) images is essential for assisting ophthalmologist in the diagnosis and grading of macular diseases. Therefore, more effective OCT volume classification for automatic recognition of macular diseases is needed. Aim: For OCT volumes in which only OCT volume-level labels are known, OCT volume classifiers based on its global feature and deep learning are designed, validated, and compared with other methods. Approach: We present a general framework to classify OCT volume for automatic recognizing macular diseases. The architecture of the framework consists of three modules: B-scan feature extractor, two-dimensional (2-D) feature map generation, and volume-level classifier. Our architecture could address OCT volume classification using two 2-D image machine learning classification algorithms. Specifically, a convolutional neural network (CNN) model is trained and used as a B-scan feature extractor to construct a 2-D feature map of an OCT volume and volume-level classifiers such as support vector machine and CNN with/without attention mechanism for 2-D feature maps are described. Results: Our proposed methods are validated on the publicly available Duke dataset, which consists of 269 intermediate age-related macular degeneration (AMD) volumes and 115 normal volumes. Fivefold cross-validation was done, and average accuracy, sensitivity, and specificity of 98.17%, 99.26%, and 95.65%, respectively, are achieved. The experiments show that our methods outperform the state-of-the-art methods. Our methods are also validated on our private clinical OCT volume dataset, consisting of 448 AMD volumes and 462 diabetic macular edema volumes. Conclusions: We present a general framework of OCT volume classification based on its 2-D feature map and CNN with attention mechanism and describe its implementation schemes. Our proposed methods could classify OCT volumes automatically and effectively with high accuracy, and they are a potential practical tool for screening of ophthalmic diseases from OCT volume. [ABSTRACT FROM AUTHOR]

• Proposing a framework to assess the emission reduction potential of ride-sharing from empirical mobility data. • Proposing a deep learning-based method that shows higher performance than traditional simulation method. • Over 1 million historical individual GPS records in Tokyo area are employed for analysis. • We found that averagely 26.97% of traveling distance can be saved and maximally 84.52% of reduction of CO2 emission. Spreading green and low-consumption transportation methods is becoming an urgent priority. Ride-sharing, which refers to the sharing of car journeys so that more than one person travel in a car, and prevents the need for others to drive to a location themselves, is a critical solution to this issue. Before being introduced into one place, it needs a potential analysis. However, current studies did this kind of analysis based on home and work locations or social ties between people, which is not precise and straight enough. Few pieces of research departed from real mobility data, but uses time-consuming methodology. In this paper, we proposed an analysis framework to bridge this gap. We chose the case study of Tokyo area with over 1 million GPS travel records and trained a deep learning model to find out this potential. From the computation result, on average, nearly 26.97% of travel distance could be saved by ride-sharing, which told us that there is a significant similarity in the travel pattern of people in Tokyo and there is considerable potential of ride-sharing. Moreover, if half of the original public transit riders in our study case adopt ride-sharing, the quantity of CO 2 is estimated to be reduced by 84.52%; if all of the original public transit riders in our study case adopt ride-sharing, 83.56% of CO 2 emission reduction can be expected with a rebound effect because of increase of participants from public transit. Ride-sharing can not only improve the air quality of these center business districts but also alleviate some city problems like traffic congestion. We believe the analysis of the potential of ride-sharing can provide insight into the decision making of ride-sharing service providers and decision-makers. [ABSTRACT FROM AUTHOR]

Pedestrian trajectory prediction under crowded circumstances is a challenging problem owing to human interaction and the complexity of the trajectory pattern. Various methods have been proposed for solving this problem, ranging from traditional Bayesian analysis to Social Force model and deep learning methods. However, most existing models heavily depend on specific scenarios because the trajectory model is constructed in absolute coordinates even though the motion trajectory as well as human interaction are in relative motion. In this study, a novel trajectory prediction model is proposed to capture the relative motion of pedestrians in extremely crowded scenarios. Trajectory sequences and human interaction are first represented with relative motion and then integrated to our model to predict pedestrians' trajectories. The proposed model is based on Long Short Term Memory (LSTM) structure and consists of an encoder and a decoder which are trained by truncated back propagation. In addition, an anisotropic neighborhood setting is proposed instead of traditional neighborhood analysis. The proposed approach is validated using trajectory data acquired at an extremely crowded train station in Tokyo, Japan. The trajectory prediction experiments demonstrated that the proposed method outperforms existing methods and is stable for predictions of varying length even when the model is trained with a controlled short trajectory sequence. [ABSTRACT FROM AUTHOR]

Incorrect predictions or underestimation of a city's solar potential can result from neglecting common features of photovoltaic (PV) panels from remote sensing images. This paper proposes an improved approach to address the challenge of accurately segmenting PV panels from remote sensing images using deep learning methods. The proposed method incorporates common features of PV panels and a constraint refinement module (CRM) to perform the localization of PV panel regions and shape regularization more accurately. Specifically, the method uses a color loss function based on prior knowledge of color to refine the predicted region with correct color information among confusing objectives, and a shape loss function based on multi-layer shape targets calculation to refine the initial segments and constrain the edge information of predicted regions. Different CRMs are embedded into the four refined initial segment modules, respectively, to improve the detection IoU of PV panels by up to 7.44%. The best CRM-integrated model performs the best IoU of 74.66% when segmenting PV panels. The proposed method has important implications for urban PV panel segmentation at the city level and provides a promising solution for remote sensing image-based PV plate segmentation tasks in challenging scenarios. • We proposed an improved feature-oriented segmentation method to enhance the accuracy and contextual information of PV panels. • We incorporated common features of PV panels and CRM to perform PV panel region localization and shape regularization. • We designed a color constraint refinement module via prior knowledge to refine the predicted region among confusing targets. • We designed a shape constraint refinement module based on multi-layer shape targets to constrain the edge of predicted areas. [ABSTRACT FROM AUTHOR]

The increasing need to develop renewable energy sources to combat climate change has led to a significant rise in demand for photovoltaic (PV) installations. Consequently, accurately detecting and estimating the capacity and potential for electricity generation of these installed PV systems has become crucial for effective energy management. However, due to the limitation of localized mechanism in convolution operations, traditional FCN-based methods face challenges in capturing global dependencies among remote sensing image patches. As a result, they struggle to model long-range interactions between different PV panels with diverse structures, including varying size, shape, and texture, particularly in distributed residential areas. To tackle the challenge of modeling PV panels with diverse structures, we propose a coupled U-Net and Vision Transformer model named TransPV for refining PV semantic segmentation. Specifically, Mix Transformer block is incorporated in the encoder to enhance the modeling of global context, while the U-Shaped structure enables the combination of multi-level features, resulting in enriched feature representation, which helps the model better understand objects in the images, regardless of their shape, size, and texture. In addition, we implement the PointRend module in the decoder to obtain finer segmentation boundary details, and utilize a novel refiner loss function during the training process to alleviate the problem of extremely unbalanced samples. The experimental results from the Heilbronn datasets demonstrate the remarkable performance of our proposed TransPV model in addressing intra-class diversity of PV structures, surpassing previous state-of-the-art methods with an impressive IoU and accuracy of 0.802 and 0.876, respectively. Furthermore, our model exhibits high generalization capability on the BDPV dataset with IoU of 0.745. The obtained results highlight the superiority of TransPV in improving accuracy and addressing diverse structure issues in PV segmentation, which provides valuable insights for optimizing the efficiency and sustainability of renewable energy. • The main challenges in real-world PV segmentation scenarios are studied. • The Vision Transformer mechanism is pioneeringly applied to enhance PV segmentation. • A novel U-shaped Vision Transformer model is designed for PV segmentation refinement. • Our method showcases remarkable segmentation accuracy and generalization capability. [ABSTRACT FROM AUTHOR]

Recently, clean solar energy has aroused wide attention due to its excellent potential for electricity production. A highly accurate prediction of photovoltaic power generation (PVPG) is the basis of the production and transmission of electricity. However, the current works neglect the regional correlation characteristics of PVPG and few studies propose an effective framework by incorporating prior knowledge for more physically reasonable results. In this work, a hybrid deep learning framework is proposed for simultaneously capturing the spatial correlations among different regions and temporal dependency patterns with various importance. The scientific theory and domain knowledge are incorporated into the deep learning model to make the predicted results possess physical reasonability. Subsequently, the theory-guided and attention-based CNN-LSTM (TG-A-CNN-LSTM) is constructed for PVPG prediction. In the training process, data mismatch and boundary constraint are incorporated into the loss function, and the positive constraint is utilized to restrict the output of the model. After receiving the parameters of the neural network, a TG-A-CNN-LSTM model, whose predicted results obey the physical law, is constructed. A real energy system in five regions is used to verify the accuracy of the proposed model. The predicted results indicate that TG-A-CNN-LSTM can achieve higher precision of PVPG prediction than other prediction models, with RMSE being 11.07, MAE being 4.98, and R 2 being 0.94, respectively. Moreover, the performance of prediction models with sparse data is tested to illustrate the stability and robustness of TG-A-CNN-LSTM. • The weight-based temporal-spatial network is utilized to capture variables correlations features. • Scientific theory is incorporated for photovoltaic power generation predicting with physical reasonability. • A multi-region real power system is utilized to illustrate the superiority of the proposed model. • The proposed model outperforms other methods in predicting photovoltaic power generation with sparse data. [ABSTRACT FROM AUTHOR]