Search

Background: Previous studies in medical imaging have shown disparate abilities of artificial intelligence (AI) to detect a person's race, yet there is no known correlation for race on medical imaging that would be obvious to human experts when interpreting the images. We aimed to conduct a comprehensive evaluation of the ability of AI to recognise a patient's racial identity from medical images., Methods: Using private (Emory CXR, Emory Chest CT, Emory Cervical Spine, and Emory Mammogram) and public (MIMIC-CXR, CheXpert, National Lung Cancer Screening Trial, RSNA Pulmonary Embolism CT, and Digital Hand Atlas) datasets, we evaluated, first, performance quantification of deep learning models in detecting race from medical images, including the ability of these models to generalise to external environments and across multiple imaging modalities. Second, we assessed possible confounding of anatomic and phenotypic population features by assessing the ability of these hypothesised confounders to detect race in isolation using regression models, and by re-evaluating the deep learning models by testing them on datasets stratified by these hypothesised confounding variables. Last, by exploring the effect of image corruptions on model performance, we investigated the underlying mechanism by which AI models can recognise race., Findings: In our study, we show that standard AI deep learning models can be trained to predict race from medical images with high performance across multiple imaging modalities, which was sustained under external validation conditions (x-ray imaging [area under the receiver operating characteristics curve (AUC) range 0·91-0·99], CT chest imaging [0·87-0·96], and mammography [0·81]). We also showed that this detection is not due to proxies or imaging-related surrogate covariates for race (eg, performance of possible confounders: body-mass index [AUC 0·55], disease distribution [0·61], and breast density [0·61]). Finally, we provide evidence to show that the ability of AI deep learning models persisted over all anatomical regions and frequency spectrums of the images, suggesting the efforts to control this behaviour when it is undesirable will be challenging and demand further study., Interpretation: The results from our study emphasise that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging., Funding: National Institute of Biomedical Imaging and Bioengineering, MIDRC grant of National Institutes of Health, US National Science Foundation, National Library of Medicine of the National Institutes of Health, and Taiwan Ministry of Science and Technology., Competing Interests: Declaration of interests MG has received speaker fees for a Harvard Medical School executive education class. HT has received consulting fees from Sirona medical, Arterys, and Biodata consortium. HT also owns lightbox AI, which provides expert annotation of medical images for radiology AI. MPL has received consulting fees from Bayer, Microsoft, Phillips, and Nines. MPL also owns stocks in Nines, SegMed, and Centaur. LAC has received support to attend meetings from MISTI Global Seed Funds. ATP has received payment for expert testimony from NCMIC insurance company. ATP also has a pending institutional patent for comorbidity prediction from radiology images. All other authors declare no competing interests., (Copyright © 2022 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license. Published by Elsevier Ltd.. All rights reserved.)

Various deep learning‐based video frame interpolation methods have been proposed in the past few years, but how to generate high quality interpolated frames in videos with large motions, complex backgrounds and rich textures is still a challenging issue. To deal with this limitation, a frame interpolation method based on residual blocks and feature pyramids is proposed. U‐Net is the main architecture of our method, which can capture multi‐layer information, segment objects from the background and obtain parameters with motion information to guide frame interpolation. However, the upsampling and subsampled of U‐Net will lose important information. In order to acquire more detailed contextual information, shortcut connection is used in the encoder basic module. At the same time, feature pyramid network is employed to capture features at different scales of the decoder to improve the representation of inter‐frame spatial‐temporal features. The experimental results show that the proposed method outperform the baseline methods in both of objective and subjective evaluations on different datasets. In particular, the method has obvious advantages on datasets which contain complex background. [ABSTRACT FROM AUTHOR]

Seismic resolution enhancement is a key step for subsurface structure characterization. Although many have proposed the use of deep learning (DL) for resolution enhancement, these are typically hindered by the limitations in the application of synthetically trained networks onto real datasets. Domain adaptation (DA) offers an approach to reduce this disparity between training and inference data, aiming through the application of data transformations to bring the distributions of both data closer to each other. We propose a simple DA procedure, termed MLReal-Lite (the light version of the earlier introduced MLReal), that mainly relies on linear operations, namely convolution and correlation; these transformations introduce aspects of the field data into the synthetic data prior to training, and vice-versa with regard to the inference stage. Taking 1-D and 2-D resolution enhancement tasks as examples, we show how the inclusion of MLReal-Lite improves the performance of neural networks. Not only do the results demonstrate notable improvements in seismic resolution, they also exhibit a higher signal-to-noise ratio (SNR) and better continuity of events, in comparison to the tests without MLReal-Lite. Finally, while illustrated on a resolution enhancement task, our proposed methodology is applicable for any seismic data of dimensions N-D, offering a DA applicable from well ties through to 3-D seismic volumes, and beyond. [ABSTRACT FROM AUTHOR]

The coronavirus disease 2019 (COVID-19) pandemic has caused millions of deaths around the world and revealed the need for data-driven models of pandemic spread. Accurate pandemic caseload forecasting allows informed policy decisions on the adoption of non-pharmaceutical interventions (NPIs) to reduce disease transmission. Using COVID-19 as an example, we present Pandemic conditional Ordinary Differential Equation (PAN-cODE), a deep learning method to forecast daily increases in pandemic infections and deaths. By using a deep conditional latent variable model, PAN-cODE can generate alternative caseload trajectories based on alternate adoptions of NPIs, allowing stakeholders to make policy decisions in an informed manner. PAN-cODE also allows caseload estimation for regions that are unseen during model training. We demonstrate that, despite using less detailed data and having fully automated training, PAN-cODE's performance is comparable to state-of-the-art methods on 4-week-ahead and 6-week-ahead forecasting. Finally, we highlight the ability of PAN-cODE to generate realistic alternative outcome trajectories on select US regions. [ABSTRACT FROM AUTHOR]

Operational monitoring of pipelines can prevent environmental and economic losses. However, pipeline data have the characteristics of high dimension and nonlinear coupling, which makes it difficult to determine the relationship between the data and process, resulting in a high rate of misjudgment of the operating condition. To address this issue, an operating condition recognition model based on kernel principal component analysis (KPCA)-convolutional neural network (CNN) is proposed. Deeppipe refers to the use of deep learning algorithms to solve pipeline-related problems. Considering the spatial and time-series characteristics of the pipeline, the inlet and outlet pressure matrixes of the initial station, intermediate station, and terminal station are constructed. Subsequently, the features of the pressure matrix in the time domain, frequency domain, and energy domain are extracted. KPCA is employed to obtain the reconstructed feature matrix, which is used as the input of the proposed CNN recognition model. Taking two multiproduct pipelines as examples, the effectiveness of the KPCA-CNN recognition model is verified while compared with traditional nonlinear classification models (e.g., artificial neural network, decision tree, random forest, and others). The results show that the proposed model has the highest accuracy, precision, recall, and F1 score, and all reach 100%, which has a certain guiding significance for the monitoring and management of onsite pipelines. [ABSTRACT FROM AUTHOR]

Personalized cancer treatments based on the molecular profile of a patient's tumor are an emerging and exciting class of treatments in oncology. As genomic tumor profiling is becoming more common, targeted treatments for specific molecular alterations are gaining traction. To discover new potential therapeutics that may apply to broad classes of tumors matching some molecular pattern, experimentalists and pharmacologists rely on high-throughput, in vitro screens of many compounds against many different cell lines. We propose a hierarchical Bayesian model of how cancer cell lines respond to drugs in these experiments and develop a method for fitting the model to real-world high-throughput screening data. Through a case study, the model is shown to capture nontrivial associations between molecular features and drug response, such as requiring both wild type TP53 and overexpression of MDM2 to be sensitive to Nutlin-3(a). In quantitative benchmarks, the model outperforms a standard approach in biology, with $\approx20\%$ lower predictive error on held out data. When combined with a conditional randomization testing procedure, the model discovers markers of therapeutic response that recapitulate known biology and suggest new avenues for investigation. All code for the article is publicly available at https://github.com/tansey/deep-dose-response. [ABSTRACT FROM AUTHOR]

Building rooftop segmentation using deep learning techniques is a popular yet challenging area of research in computer vision and remote sensing image processing. While recent studies have developed various deep learning models, there has been less focus on investigating the basic elements of network architecture and foreground-background balance in sampled images. To address this research gap, this study proposes optimizing the UNet basic elements and image foreground-background balance to improve building rooftop segmentation accuracy. The Inria dataset is used for model training and accuracy evaluation. The results show that the impact of network backbone on segmentation accuracy depends on its ability to extract lower-level features and whether it is pretrained by large amounts of data. This study also finds that residual structures of network backbone can negatively affect performance regardless of the network depth. Among the decoding network upsampling strategies, the proposed super-resolution method achieves the best segmentation results, but does not significantly differ from the commonly used bilinear interpolation and deconvolution methods. Additionally, this study proposes a hybrid loss function that considers area, arithmetic, shape, spectral, and texture discrepancy, which further improves the segmentation performance. Finally, this study highlights the significant impact of image foreground-background balance on building rooftop segmentation and proposes an ensemble model to mitigate image foreground-background imbalance and improve segmentation performance. [ABSTRACT FROM AUTHOR]

Video frame interpolation is used to generate intermediate frames by estimating the movement of pixels between the input frames. However, problems of blurring, object occlusion, and sudden brightness changes occur in naturally obtained video frames. We propose a context-based video frame interpolation method via depthwise over-parameterized convolution. First, the proposed network obtains the context graphs of the input frames. Subsequently, an adaptive collaboration of flows is adopted to warp the input frames and the context graphs. Then, the frame synthesis network is used to fuse the warped input frames and context graphs to obtain a preliminary estimate of the interpolated frame. Finally, a post-processing module is employed to refine the result. Experimental results on several datasets demonstrate that the proposed method performs qualitatively and quantitatively better than state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Leak fault significantly affects the reliable and sustainable green hydrogen energy supply by renewable Power-to-Hydrogen (P2H2) system. Deep learning has been widely applied to automated leak fault detection and diagnosis of hydrogen systems. However, due to the intermittency of wind/solar-power-generation, existing approaches developed by monitored signals have the bad generalization to leak scenario under large volatility of power-to‑hydrogen production. This study introduces graph deep probability learning approach, in which an attention-based graph neural network (GNN) learns the dependency between installed sensors. Variational inference is integrated to model posterior distribution of sensor dependencies, by using which the leak fault under large volatility of power-to‑hydrogen production is detected and localized by using normal time-series signals under different timeslots of one day as training data. Experiment of hydrogen leak faults from P2H2 system is conducted to verify the more accuracy of proposed approach compared to 6 state-of-the-art approaches. Results demonstrated our proposed approach detects the leak fault more accurately with a higher AUC of 0.96 and successfully localizes all the leak faults. This study supports more reliable and efficient safety monitoring for upcoming renewable P2H2 system in future. • A graph deep probability learning approach is proposed to accurately detect and localize leak fault of P2H2 production. • Experiment of hydrogen leak is conducted to compare our proposed approach with 6 existing approaches. • Our approach achieves higher detection accuracy and localizes all faults when large volatility of P2H2 production exists. • Our proposed approach has better generalization to faults detection and diagnosis of upcoming renewable P2H2 system. [ABSTRACT FROM AUTHOR]

Particle identification and shape evaluation of granular materials from their realistic packing images are challenging and of great interest to many engineers and researchers. In this study, a systematic tool is developed based on computing techniques, including deep learning and computational geometry. First, image datasets of the target granular particles with well-labeled masks are established. The Mask Region Convolutional Neural Network (Mask R-CNN) is employed to implement the end-to-end instance segmentation and contour extraction of particles on different realistic images. Since Mask R-CNN models have several different feature extraction backbones, the optimal model is selected and then trained on the established datasets using transfer learning technique. After the particles are successfully identified from images of cobble and ballast, the elongation, angularity, and roughness are evaluated and the statistical shape analysis is conducted. The proposed method has strong generalization ability, especially for densely-packed particles. [Display omitted] • Automated particle identification using the Mask R-CNN model. • Successful instance segmentation of densely-packed granular particles. • Application to automated shape evaluation of ballast and cobble samples. [ABSTRACT FROM AUTHOR]

Incorrect predictions or underestimation of a city's solar potential can result from neglecting common features of photovoltaic (PV) panels from remote sensing images. This paper proposes an improved approach to address the challenge of accurately segmenting PV panels from remote sensing images using deep learning methods. The proposed method incorporates common features of PV panels and a constraint refinement module (CRM) to perform the localization of PV panel regions and shape regularization more accurately. Specifically, the method uses a color loss function based on prior knowledge of color to refine the predicted region with correct color information among confusing objectives, and a shape loss function based on multi-layer shape targets calculation to refine the initial segments and constrain the edge information of predicted regions. Different CRMs are embedded into the four refined initial segment modules, respectively, to improve the detection IoU of PV panels by up to 7.44%. The best CRM-integrated model performs the best IoU of 74.66% when segmenting PV panels. The proposed method has important implications for urban PV panel segmentation at the city level and provides a promising solution for remote sensing image-based PV plate segmentation tasks in challenging scenarios. • We proposed an improved feature-oriented segmentation method to enhance the accuracy and contextual information of PV panels. • We incorporated common features of PV panels and CRM to perform PV panel region localization and shape regularization. • We designed a color constraint refinement module via prior knowledge to refine the predicted region among confusing targets. • We designed a shape constraint refinement module based on multi-layer shape targets to constrain the edge of predicted areas. [ABSTRACT FROM AUTHOR]

The effective segmentation of esophageal squamous carcinoma lesions in CT scans is significant for auxiliary diagnosis and treatment. However, accurate lesion segmentation is still a challenging task due to the irregular form of the esophagus and small size, the inconsistency of spatio-temporal structure, and low contrast of esophagus and its peripheral tissues in medical images. The objective of this study is to improve the segmentation effect of esophageal squamous cell carcinoma lesions. It is critical for a segmentation network to effectively extract 3D discriminative features to distinguish esophageal cancers from some visually closed adjacent esophageal tissues and organs. In this work, an efficient HRU-Net architecture (High-Resolution U-Net) was exploited for esophageal cancer and esophageal carcinoma segmentation in CT slices. Based on the idea of localization first and segmentation later, the HRU-Net locates the esophageal region before segmentation. In addition, an Resolution Fusion Module (RFM) was designed to integrate the information of adjacent resolution feature maps to obtain strong semantic information, as well as preserve the high-resolution features. Compared with the other five typical methods, the devised HRU-Net is capable of generating superior segmentation results. Our proposed HRU-NET improves the accuracy of segmentation for squamous esophageal cancer. Compared to other models, our model performs the best. The designed method may improve the efficiency of clinical diagnosis of esophageal squamous cell carcinoma lesions. [ABSTRACT FROM AUTHOR]

The abilities to understand pedestrian social interaction behaviors and to predict their future trajectories are critical for road safety, traffic management and more broadly autonomous vehicles and robots. Social interactions are intuitively heterogeneous and dynamic over time and circumstances, making them hard to explain. In this paper, we creatively investigate modeling interpretable social interactions for pedestrian trajectory, which is not considered by the existing trajectory prediction research. Moreover, we propose a two-stage methodology for interaction modeling - "mode extraction" and "mode aggregation", and develop a long short-term memory (LSTM)-based model for long-term trajectory prediction, which naturally takes into account multi-types of social interactions. Different from previous models that do not explain how pedestrians interact socially, we extract latent modes that represent social interaction types which scales to an arbitrary number of neighbors. Extensive experiments over two public datasets have been conducted. The quantitative and qualitative results demonstrate that our method is able to capture the multi-modality of human motion and achieve better performance under specific conditions. Its performance is also verified by the interpretation of predicted modes, of which the results are in accordance with common sense. Besides, we have performed sensitivity analysis on the crucial hyperparameters in our model. Code is available at: https://github.com/xiaoluban/Modeling-Interpretable-Social-Interactions-for-Pedestrian-Trajectory. • People in crowds interacts with neighbors following different patterns. • Incorporating heterogeneous interaction modeling into trajectory prediction problem. • Extracting latent modes which scale to arbitrary number of neighbors. • Improving reliability of the prediction model with interpretation of extracted modes. [ABSTRACT FROM AUTHOR]

• A Transformer-based data-driven model is developed to predict CO 2 concentration and FG flowrate for a waste fired CHP plant. • The impact of different meteorological parameters is investigated as input features. • The Transformer model can give better accuracy than LSTM and ANN. • Including NIR spectral data can improve the accuracy of the Transformer model. Integrating CO 2 capture with biomass-fired combined heat and power (bio-CHP) plants is a promising method to achieve negative emissions. However, the use of versatile biomass, including waste, and the dynamic operation of bio-CHP plants leads to large fluctuations in the flowrate and CO 2 concentration of the flue gas (FG), which further affect the operation of post-combustion CO 2 capture. To optimize the dynamic operation of CO 2 capture, a reliable model to predict the FG flowrate and CO 2 concentration in real time is essential. In this paper, a data-driven model based on the Transformer architecture is developed. The model validation shows that the root mean squared error (RMSE), mean absolute percentage error (MAPE), and Pearson correlation coefficient (PPMCC) of Transformer are 0.3553, 0.0189, and 0.8099 respectively for the prediction of FG flowrate; and 13.137, 0.0318, and 0.8336 respectively for the prediction of CO 2 concentration. The potential impact of various meteorological parameters on model accuracy is also assessed by analyzing the Shapley value. It is found that temperature and direct horizontal irradiance (DHI) are the most important factors, which should be selected as input features. In addition, using the near-infrared (NIR) spectral data as input features is also found to be an effective way to improve the prediction accuracy. It can reduce RMSE and MAPE for CO 2 concentration from 0.2982 to 0.2887 and 0.0158 to 0.0157 respectively, and RMSE and MAPE for FG flowrate from 4.9854 to 4.7537 and 0.0141 to 0.0121 respectively. The Transformer model is also compared to other models, including long short-term memory network (LSTM) and artificial neural network (ANN), which results show that the Transformer model is superior in predicting complex dynamic patterns and nonlinear relationships. [ABSTRACT FROM AUTHOR]

The increasing need to develop renewable energy sources to combat climate change has led to a significant rise in demand for photovoltaic (PV) installations. Consequently, accurately detecting and estimating the capacity and potential for electricity generation of these installed PV systems has become crucial for effective energy management. However, due to the limitation of localized mechanism in convolution operations, traditional FCN-based methods face challenges in capturing global dependencies among remote sensing image patches. As a result, they struggle to model long-range interactions between different PV panels with diverse structures, including varying size, shape, and texture, particularly in distributed residential areas. To tackle the challenge of modeling PV panels with diverse structures, we propose a coupled U-Net and Vision Transformer model named TransPV for refining PV semantic segmentation. Specifically, Mix Transformer block is incorporated in the encoder to enhance the modeling of global context, while the U-Shaped structure enables the combination of multi-level features, resulting in enriched feature representation, which helps the model better understand objects in the images, regardless of their shape, size, and texture. In addition, we implement the PointRend module in the decoder to obtain finer segmentation boundary details, and utilize a novel refiner loss function during the training process to alleviate the problem of extremely unbalanced samples. The experimental results from the Heilbronn datasets demonstrate the remarkable performance of our proposed TransPV model in addressing intra-class diversity of PV structures, surpassing previous state-of-the-art methods with an impressive IoU and accuracy of 0.802 and 0.876, respectively. Furthermore, our model exhibits high generalization capability on the BDPV dataset with IoU of 0.745. The obtained results highlight the superiority of TransPV in improving accuracy and addressing diverse structure issues in PV segmentation, which provides valuable insights for optimizing the efficiency and sustainability of renewable energy. • The main challenges in real-world PV segmentation scenarios are studied. • The Vision Transformer mechanism is pioneeringly applied to enhance PV segmentation. • A novel U-shaped Vision Transformer model is designed for PV segmentation refinement. • Our method showcases remarkable segmentation accuracy and generalization capability. [ABSTRACT FROM AUTHOR]

The pressure changes dramatically during the shutdown process of the multi-product pipeline. When the pipeline pressure comes to decrease, it is often mistaken as pipeline leakage or other abnormal condition which increases the burden of the operator on-site. At present, the method of pipeline shutdown pressure analysis is mainly based on numerical simulation which can not monitor shutdown pressure in real-time. In this work, the time-series approximate ability of long short-term memory (LSTM) is taken advantage of to construct a shutdown pressure prediction model. To overcome the drawback of this deep learning algorithm that is trained only by ample data, the scientific principle and theory are integrated into LSTM. Subsequently, the theory-guided long short-term memory (TG-LSTM) is proposed for pipeline shutdown pressure prediction. The proposed model is trained with available data and simultaneously guided by the theory (physical principle and engineering theory) of the underlying problem. In the training process, the data mismatch, as well as monotonicity constraints, and boundary constraints are coupled into loss function. After acquiring the parameters of the neural network, a TG-LSTM model is established which not only fits the data, but also follows the physical principle and the engineering theory. The proposed model is verified by three real-world multi-product pipelines. The results indicate that TG-LSTM achieves better accuracy than other prediction models, with MAPE being 0.246%, 0.186%, and 0.143%, respectively. Finally, the sensitivity analysis of different hyper-parameter is conducted to illustrate the robustness of TG-LSTM in pipeline shutdown pressure prediction. [ABSTRACT FROM AUTHOR]

The widespread adoption of photovoltaic (PV) technology for renewable energy necessitates accurate segmentation of PV panels to estimate installation capacity. However, achieving highly efficient and precise segmentation methods remains a pressing challenge. Recent advancements in artificial intelligence and remote sensing techniques have shown promise in PV segmentation. Nevertheless, real-world scenarios introduce complexities such as diverse sensing platforms, sensors, panel categories, and testing regions. These factors contribute to resolution, size, and foreground-background class imbalances, impeding accurate and generalized PV panel segmentation over large areas. To address these challenges, we propose GenPV, a deep learning model that leverages data distribution analysis and PV panel characteristics to enhance segmentation accuracy and generalization. GenPV employs a multi-scale feature learning approach, utilizing an enhanced feature pyramid network to fuse data features from multiple resolutions, effectively addressing resolution imbalance. Moreover, inductive learning is employed through a multitask approach, facilitating the detection and identification of both small and large-sized PV panels to mitigate size imbalance. To address significant class imbalance in PV panel recognition tasks, we integrate the Focal loss function for effective hard sample mining. Through experimental evaluation conducted in Heilbronn, Germany, our proposed method demonstrates superior performance compared to state-of-the-art approaches in PV panel segmentation. The results exhibit progressively higher accuracy and improved generalization capability. These findings highlight the potential of our method to serve as an advanced and practical tool for PV segmentation in the renewable energy field. • Analysis of the imbalance problems in real-world PV semantic segmentation scenarios. • Creation of an open-source PV panel dataset to advance renewable energy development. • Introducing a novel end-to-end DL model named GenPV for PV panel segmentation. • Improved accuracy and generalization in PV segmentation across unaligned datasets. [ABSTRACT FROM AUTHOR]

Recently, clean solar energy has aroused wide attention due to its excellent potential for electricity production. A highly accurate prediction of photovoltaic power generation (PVPG) is the basis of the production and transmission of electricity. However, the current works neglect the regional correlation characteristics of PVPG and few studies propose an effective framework by incorporating prior knowledge for more physically reasonable results. In this work, a hybrid deep learning framework is proposed for simultaneously capturing the spatial correlations among different regions and temporal dependency patterns with various importance. The scientific theory and domain knowledge are incorporated into the deep learning model to make the predicted results possess physical reasonability. Subsequently, the theory-guided and attention-based CNN-LSTM (TG-A-CNN-LSTM) is constructed for PVPG prediction. In the training process, data mismatch and boundary constraint are incorporated into the loss function, and the positive constraint is utilized to restrict the output of the model. After receiving the parameters of the neural network, a TG-A-CNN-LSTM model, whose predicted results obey the physical law, is constructed. A real energy system in five regions is used to verify the accuracy of the proposed model. The predicted results indicate that TG-A-CNN-LSTM can achieve higher precision of PVPG prediction than other prediction models, with RMSE being 11.07, MAE being 4.98, and R 2 being 0.94, respectively. Moreover, the performance of prediction models with sparse data is tested to illustrate the stability and robustness of TG-A-CNN-LSTM. • The weight-based temporal-spatial network is utilized to capture variables correlations features. • Scientific theory is incorporated for photovoltaic power generation predicting with physical reasonability. • A multi-region real power system is utilized to illustrate the superiority of the proposed model. • The proposed model outperforms other methods in predicting photovoltaic power generation with sparse data. [ABSTRACT FROM AUTHOR]

• Theory-guided HCP is proposed to introduce domain knowledge and prior information as hard constraints into neural networks through projection. • A projection matrix is constructed based on the governing equation, which can be regarded as a non-trainable activation function in the neural networks. • Experiment shows that hard constraint can effectively reduce data demand and increase prediction accuracy with sparse and partial observations. Machine learning models have been successfully used in many scientific and engineering fields. However, it remains difficult for a model to simultaneously utilize domain knowledge and experimental observation data. The application of knowledge-based symbolic artificial intelligence (AI) represented by expert systems is limited by the expressive ability of the model, and data-driven connectionism AI represented by neural networks is prone to produce predictions that might violate physical principles. In order to fully integrate domain knowledge with observations and make full use of the strong fitting ability of neural networks, this study proposes theory-guided hard constraint projection (HCP). This deep learning model converts physical constraints, such as governing equations, into a form that is easy to handle through discretization, and then implements hard constraint optimization through projection in a patch. Based on rigorous mathematical proofs, theory-guided HCP can ensure that model predictions strictly conform to physical mechanisms in the constraint patch. The training process of theory-guided HCP only needs a small amount of labeled data (sparse observation), and it can supervise the model by combining the coordinates (label-free data) with domain knowledge. The performance of the theory-guided HCP is verified by experiments based on a published heterogeneous subsurface flow problem. The experiments show that theory-guided HCP requires fewer data, and achieves higher prediction accuracy and stronger robustness to noisy observations, than the fully connected neural networks and soft constraint models. Furthermore, due to the application of domain knowledge, theory-guided HCP possesses the ability to extrapolate and can accurately predict points outside of the range of the training dataset. [ABSTRACT FROM AUTHOR]