Search

Pneumonia remains a global health issue, creating the need for accurate detection methods for effective treatment. Deep learning models like ResNet50 show promise in detecting pneumonia from chest X-rays; however, their black-box nature limits the transparency, which fails to meet that needed for clinical trust. This study aims to improve model interpretability by comparing four interpretability techniques, which are Layer-wise Relevance Propagation (LRP), Adversarial Training, Class Activation Maps (CAMs), and the Spatial Attention Mechanism, and determining which fits best the model, enhancing its transparency with minimal impact on its performance. Each technique was evaluated for its impact on the accuracy, sensitivity, specificity, AUC-ROC, Mean Relevance Score (MRS), and a calculated trade-off score that balances interpretability and performance. The results indicate that LRP was the most effective in enhancing interpretability, achieving high scores across all metrics without sacrificing diagnostic accuracy. The model achieved 0.91 accuracy and 0.85 interpretability (MRS), demonstrating its potential for clinical integration. In contrast, Adversarial Training, CAMs, and the Spatial Attention Mechanism showed trade-offs between interpretability and performance, each highlighting unique image features but with some impact on specificity and accuracy. [ABSTRACT FROM AUTHOR]

Soil salinization affects agricultural productivity and ecosystem health in Xinjiang, especially in arid areas. The region's complex topography and limited agricultural data emphasize the pressing need for effective, large-scale monitoring technologies. Therefore, 1044 soil samples were collected from arid farmland in northern Xinjiang, and the potential effectiveness of soil salinity monitoring was explored by combining environmental variables with Landsat 8 and Sentinel-2. The study applied four types of feature selection algorithms: Random Forest (RF), Competitive Adaptive Reweighted Sampling (CARS), Uninformative Variable Elimination (UVE), and Successive Projections Algorithm (SPA). These variables are then integrated into various machine learning models—such as Ensemble Tree (ETree), Random Forest (RF), Extreme Gradient Boosting (XGBoost), and LightBoost—as well as deep learning models, including Convolutional Neural Networks (CNN), Residual Networks (ResNet), Multilayer Perceptrons (MLP), and Kolmogorov–Arnold Networks (KAN), for modeling. The results suggest that fertilizer use plays a critical role in soil salinization processes. Notably, the interpretable model KAN achieved an accuracy of 0.75 in correctly classifying the degree of soil salinity. This study highlights the potential of integrating multi-source remote sensing data with deep learning technologies, offering a pathway to large-scale soil salinity monitoring, and thereby providing valuable support for soil management. [ABSTRACT FROM AUTHOR]

As a subfield of artificial intelligence (AI), machine learning (ML) has emerged as a versatile tool in accelerating catalytic materials discovery because of its ability to find complex patterns in high‐dimensional data. While the intricacy of cutting‐edge ML models, such as deep learning, makes them powerful, it also renders decision‐making processes challenging to explain. Recent advances in explainable AI technologies, which aim to make the inner workings of ML models understandable to humans, have considerably increased our capacity to gain insights from data. In this study, taking the oxygen reduction reaction (ORR) on {111}‐oriented Pt monolayer core–shell catalysts as an example, we show how the recently developed theory‐infused neural network (TinNet) algorithm enables a rapid search for optimal site motifs with the chemisorption energy of hydroxyl (OH) as a single descriptor, revealing the underlying physical factors that govern the variations in site reactivity. By exploring a broad design space of Pt monolayer core–shell alloys (∼17,000$\sim 17,000$ candidates) that were generated from ∼1500$\sim 1500$ thermodynamically stable bulk structures in existing material databases, we identified novel alloy systems along with previously known catalysts in the goldilocks zone of reactivity properties. SHAP (SHapley Additive exPlanations) analysis reveals the important role of adsorbate resonance energies that originate from sp$sp$‐band interactions in chemical bonding at metal surfaces. Extracting physical insights into surface reactivity with explainable AI opens up new design pathways for optimizing catalytic performance beyond active sites. [ABSTRACT FROM AUTHOR]

This paper proposes a novel interpretable tourism demand forecasting framework that considers the impact of the COVID-19 pandemic by using multi-source heterogeneous data, namely, historical tourism volume, newly confirmed cases in tourist origins and destinations, and search engine data. This paper introduces newly confirmed cases in tourist origins and tourist destinations to forecast tourism demand and proposes a new two-stage decomposition method called ensemble empirical mode decomposition-variational mode decomposition to deal with the tourist arrival sequence. To solve the problem of insufficient interpretability of existing tourism demand forecasting, this paper also proposes a novel interpretable tourism demand forecasting model called JADE-TFT, which utilizes an adaptive differential evolution algorithm with external archiving (JADE) to intelligently and efficiently optimize the hyperparameters of temporal fusion transformers (TFT). The validity of the proposed prediction framework is verified by actual cases based on Hainan and Macau tourism data sets. The interpretable experimental results show that newly confirmed cases in tourist origins and tourist destinations can better reflect tourists' concerns about travel in the post-pandemic era, and the two-stage decomposition method can effectively identify the inflection point of tourism prediction, thereby increasing the prediction accuracy of tourism demand. [ABSTRACT FROM AUTHOR]

The devastating impact of COVID-19 on the United States has been profound since its onset in January 2020. Predicting the trajectory of epidemics accurately and devising strategies to curb their progression are currently formidable challenges. In response to this crisis, we propose COVINet, which combines the architecture of Long Short-Term Memory and Gated Recurrent Unit, incorporating actionable covariates to offer high-accuracy prediction and explainable response. First, we train COVINet models for confirmed cases and total deaths with five input features, and compare Mean Absolute Errors (MAEs) and Mean Relative Errors (MREs) of COVINet against ten competing models from the United States CDC in the last four weeks before April 26, 2021. The results show COVINet outperforms all competing models for MAEs and MREs when predicting total deaths. Then, we focus on prediction for the most severe county in each of the top 10 hot-spot states using COVINet. The MREs are small for all predictions made in the last 7 or 30 days before March 23, 2023. Beyond predictive accuracy, COVINet offers high interpretability, enhancing the understanding of pandemic dynamics. This dual capability positions COVINet as a powerful tool for informing effective strategies in pandemic prevention and governmental decision-making. [ABSTRACT FROM AUTHOR]

The formation of floods, as a complex physical process, exhibits dynamic changes in its driving factors over time and space under climate change. Due to the black‐box nature of deep learning, its use alone does not enhance understanding of hydrological processes. The challenge lies in employing deep learning to uncover new knowledge on flood formation mechanism. This study proposes an interpretable framework for deep learning flood modeling that employs interpretability techniques to elucidate the inner workings of a peak‐sensitive Informer, revealing the dynamic response of floods to driving factors in 482 watersheds across the United States. Accurate simulation is a prerequisite for interpretability techniques to provide reliable information. The study reveals that comparing the Informer with Transformer and LSTM, the former showed superior performance in peak flood simulation (Nash‐Sutcliffe Efficiency over 0.6 in 70% of watersheds). By interpreting Informer's decision‐making process, three primary flood‐inducing patterns were identified: Precipitation, excess soil water, and snowmelt. The controlling effect of dominant factors is regional, and their impact on floods in time steps shows significant differences, challenging the traditional understanding that variables closer to the timing of flood event occurrence have a greater impact. Over 40% of watersheds exhibited shifts in dominant driving factors between 1981 and 2020, with precipitation‐dominated watersheds undergoing more significant changes, corroborating climate change responses. Additionally, the study unveils the interplay and dynamic shifts among variables. These findings suggest that interpretable deep learning, through reverse deduction, transforms data‐driven models from merely fitting nonlinear relationships to effective tools for enhancing understanding of hydrological characteristics. Plain Language Summary: The formation of floods is often dynamically influenced by multiple driving factors. Traditional methods based on statistics or hydrological models struggle to clearly understand flood mechanisms due to their limitations. Although deep learning has become an effective tool for flood modeling, its black‐box nature makes it difficult to enrich understanding of flood processes through it. Our proposed interpretable deep learning offers a perspective to unveil the dynamic drivers of floods by extracting patterns from deep learning in a reverse deduction approach. We started with models that perform best on flood extremes and identified the dominant factors and variations of floods across 482 watersheds in the United States. These variations exhibit regional characteristics, with precipitation playing a more significant role and showing more pronounced trends in watersheds where floods are primarily driven by precipitation. We found that variables from 2 days before a flood could have a greater impact than those from the previous days. Furthermore, we revealed the combined impact of variables on flooding through deep learning, showcasing their dynamic changes. Interpretable deep learning explores a new avenue to derive new hydrological insights from inverse data, helping hydrologists better understand natural physical processes. Key Points: Proposed a framework for interpretable time series deep learning based on optimal performance to reveal the dynamic drivers of floodsUnveiled the changes in the dominant driving factors of floods and the roles of driving factors at different moments before a floodElucidated the dynamic responses of floods to the interactions among driving factors [ABSTRACT FROM AUTHOR]

Sustainable Development Goal 7 is dedicated to ensuring access to clean and affordable energy that can be utilized in various applications. Solar panels (SP) are utilized to convert sunlight into electricity, acting as a renewable energy source. It is important to keep SP clean to obtain the required performance, as the accumulation of snow and dust on SP greatly affects the amount of electricity generated. On the other hand, excessive cleaning has some detrimental effects on the SP, therefore cleaning should only be done when necessary and not on a regular basis. Consequently, it is critical to determine whether the cleaning procedure is necessary by automatically detecting the presence of dust or snow on the panels while avoiding inaccurate predictions. Research efforts have been made to detect the presence of dust and snow on SP, but most of the proposed methods do not guarantee accurate detection results. This paper proposes an accurate, reliable, and interpretable approach called Solar-OBNet. The proposed Solar-OBNet can detect dusty SP and snow-covered SP very efficiently and be used in conjunction with the methods used to clean SP. The proposed Solar-OBNet is based on a Bayesian convolutional neural network, which enables it to express the amount of confidence in its predictions. Two measurements are used to estimate the uncertainty in the outcomes of the proposed Solar-OBNet, namely predictive entropy and standard deviation. The proposed Solar-OBNet can express confidence in the correct predictions by showing low values for predictive entropy and standard deviation. The proposed Solar-OBNet can also give an uncertainty warning in the case of erroneous predictions by showing high values of predictive entropy and standard deviation. The proposed Solar-OBNet's efficacy was verified by interpreting its results using a method called Weighted Gradient-Directed Class Activation Mapping (Grad-CAM). The proposed Solar-OBNet has achieved a balanced accuracy of 94.07% and an average specificity 95.83%, outperforming other comparable methods. [ABSTRACT FROM AUTHOR]

Accurately predicting traffic accident severity is crucial for road safety. However, existing studies lack interpretability in revealing the relationship between accident severity and key factors. To address this issue, we propose a new interpretable analytical framework. The framework utilizes XGBoost and SHAP to select effective factors. Then the AISTGCN model is constructed by improving the STGCN through the local attention mechanism to predict the severity of the accident. Finally, DeepLIFT is used to interpret the forecasts and identify key factors. Our experiments using real-world UK accident data demonstrate that our proposed AISTGCN outperforms baseline models in outcome prediction with an accuracy of 0.8772. The computation time was reduced and the reliability of predictions was enhanced through screening for effective factors. Furthermore, DeepLIFT provides more reasonable explanations when explaining accidents of different severity, indicating that vehicle count significantly impacts. Our framework aids in developing effective safety measures to reduce accidents. [ABSTRACT FROM AUTHOR]

Tumorigenesis arises from the dysfunction of cancer genes, leading to uncontrolled cell proliferation through various mechanisms. Establishing a complete cancer gene catalogue will make precision oncology possible. Although existing methods based on graph neural networks (GNN) are effective in identifying cancer genes, they fall short in effectively integrating data from multiple views and interpreting predictive outcomes. To address these shortcomings, an interpretable representation learning framework IMVRL-GCN is proposed to capture both shared and specific representations from multiview data, offering significant insights into the identification of cancer genes. Experimental results demonstrate that IMVRL-GCN outperforms state-of-the-art cancer gene identification methods and several baselines. Furthermore, IMVRL-GCN is employed to identify a total of 74 high-confidence novel cancer genes, and multiview data analysis highlights the pivotal roles of shared, mutation-specific, and structure-specific representations in discriminating distinctive cancer genes. Exploration of the mechanisms behind their discriminative capabilities suggests that shared representations are strongly associated with gene functions, while mutation-specific and structure-specific representations are linked to mutagenic propensity and functional synergy, respectively. Finally, our in-depth analyses of these candidates suggest potential insights for individualized treatments: afatinib could counteract many mutation-driven risks, and targeting interactions with cancer gene SRC is a reasonable strategy to mitigate interaction-induced risks for NR3C1 , RXRA , HNF4A , and SP1. [ABSTRACT FROM AUTHOR]

By applying AI techniques to a variety of pandemic-relevant data, artificial intelligence (AI) has substantially supported the control of the spread of the SARS-CoV-2 virus. Along with this, epidemiological machine learning studies of SARS-CoV-2 have been frequently published. While these models can be perceived as precise and policy-relevant to guide governments towards optimal containment policies, their black box nature can hamper building trust and relying confidently on the prescriptions proposed. This paper focuses on interpretable AI-based epidemiological models in the context of the recent SARS-CoV-2 pandemic. We systematically review existing studies, which jointly incorporate AI, SARS-CoV-2 epidemiology, and explainable AI approaches (XAI). First, we propose a conceptual framework by synthesizing the main methodological features of the existing AI pipelines of SARS-CoV-2. Upon the proposed conceptual framework and by analyzing the selected epidemiological studies, we reflect on current research gaps in epidemiological AI toolboxes and how to fill these gaps to generate enhanced policy support in the next potential pandemic. [ABSTRACT FROM AUTHOR]

Sexual dimorphism in prevalence, severity and genetic susceptibility exists for most common diseases. However, most genetic and clinical outcome studies are designed in sex-combined framework considering sex as a covariate. Few sex-specific studies have analyzed males and females separately, which failed to identify gene-by-sex interaction. Here, we propose a novel unified biologically interpretable deep learning-based framework (named SPIN) for sexual dimorphism analysis. We demonstrate that SPIN significantly improved the C-index up to 23.6% in TCGA cancer datasets, and it was further validated using asthma datasets. In addition, SPIN identifies sex-specific and -shared risk loci that are often missed in previous sex-combined/-separate analysis. We also show that SPIN is interpretable for explaining how biological pathways contribute to sexual dimorphism and improve risk prediction in an individual level, which can result in the development of precision medicine tailored to a specific individual's characteristics. [ABSTRACT FROM AUTHOR]

Addressing challenges in crop growth monitoring, such as limited assessment dimensions, incomplete coverage of growth cycles, and limited deep learning (DL) interpretability, a novel dual-pol SAR rice growth monitoring method using a new crop growth index (CGI) and an interpretable DL architecture is proposed. Initially, radar vegetation indices, polarimetric decomposition parameters, and backscattering coefficients characterize rice growth in multiple dimensions. Subsequently, a CGI is designed, combining fractional vegetation cover and leaf chlorophyll content to accurately depict rice growth status, capturing both the structural and physiological activity of rice. Finally, an interpretable DL architecture incorporating a feature selection explainer and a feature-aware enhanced segmentation model is introduced. This architecture incorporates feature-wise variable learning networks, interprets the importance of individual features, and optimizes the feature composition, significantly enhancing the interpretability of the DL model. This architecture is also applicable to other neural networks. The method is applied in Suihua City, Heilongjiang Province, China, using Sentinel-1 dual-pol data from 2022 and 2023. Experiments indicate that the proposed method achieves an overall accuracy of 90.57% in the test set and a generalization accuracy of 90.43%. The integration of CGI and the interpretable DL architecture enhances the reliability of rice growth monitoring results.

Abstract The formation of floods, as a complex physical process, exhibits dynamic changes in its driving factors over time and space under climate change. Due to the black‐box nature of deep learning, its use alone does not enhance understanding of hydrological processes. The challenge lies in employing deep learning to uncover new knowledge on flood formation mechanism. This study proposes an interpretable framework for deep learning flood modeling that employs interpretability techniques to elucidate the inner workings of a peak‐sensitive Informer, revealing the dynamic response of floods to driving factors in 482 watersheds across the United States. Accurate simulation is a prerequisite for interpretability techniques to provide reliable information. The study reveals that comparing the Informer with Transformer and LSTM, the former showed superior performance in peak flood simulation (Nash‐Sutcliffe Efficiency over 0.6 in 70% of watersheds). By interpreting Informer's decision‐making process, three primary flood‐inducing patterns were identified: Precipitation, excess soil water, and snowmelt. The controlling effect of dominant factors is regional, and their impact on floods in time steps shows significant differences, challenging the traditional understanding that variables closer to the timing of flood event occurrence have a greater impact. Over 40% of watersheds exhibited shifts in dominant driving factors between 1981 and 2020, with precipitation‐dominated watersheds undergoing more significant changes, corroborating climate change responses. Additionally, the study unveils the interplay and dynamic shifts among variables. These findings suggest that interpretable deep learning, through reverse deduction, transforms data‐driven models from merely fitting nonlinear relationships to effective tools for enhancing understanding of hydrological characteristics.

Soil salinization affects agricultural productivity and ecosystem health in Xinjiang, especially in arid areas. The region’s complex topography and limited agricultural data emphasize the pressing need for effective, large-scale monitoring technologies. Therefore, 1044 soil samples were collected from arid farmland in northern Xinjiang, and the potential effectiveness of soil salinity monitoring was explored by combining environmental variables with Landsat 8 and Sentinel-2. The study applied four types of feature selection algorithms: Random Forest (RF), Competitive Adaptive Reweighted Sampling (CARS), Uninformative Variable Elimination (UVE), and Successive Projections Algorithm (SPA). These variables are then integrated into various machine learning models—such as Ensemble Tree (ETree), Random Forest (RF), Extreme Gradient Boosting (XGBoost), and LightBoost—as well as deep learning models, including Convolutional Neural Networks (CNN), Residual Networks (ResNet), Multilayer Perceptrons (MLP), and Kolmogorov–Arnold Networks (KAN), for modeling. The results suggest that fertilizer use plays a critical role in soil salinization processes. Notably, the interpretable model KAN achieved an accuracy of 0.75 in correctly classifying the degree of soil salinity. This study highlights the potential of integrating multi-source remote sensing data with deep learning technologies, offering a pathway to large-scale soil salinity monitoring, and thereby providing valuable support for soil management.

Knowledge tracing aims to model and predict students' knowledge states during learning activities. Traditional methods like Bayesian Knowledge Tracing (BKT) and logistic regression have limitations in granularity and performance, while deep knowledge tracing (DKT) models often suffer from lacking transparency. This paper proposes a Transformer-based framework that emphasizes both accuracy and interpretability. It captures the relationship between student behaviors and learning outcomes considering the associations between exam and exercise problems. We participated in the EDM Cup 2023 Contest using the proposed framework and achieved first place on the task of predicting students' performance on end-of-unit test problems using clickstream data from previous assignments. Furthermore, the framework provides meaningful insights by analyzing user actions and visualizing attention weight matrices. These insights enable targeted interventions and personalized support, enhancing online learning experiences. We have uploaded our code, saved models, and predictions to an OSF repository: https://osf.io/mdpzc/. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNN) are increasingly being used in neuroimaging research for the diagnosis of brain disorders and understanding of human brain. Despite their impressive performance, their usage in medical applications will be limited unless there is more transparency on how these algorithms arrive at their decisions. We address this issue in the current report. A DNN classifier was trained to discriminate between healthy subjects and those with posttraumatic stress disorder (PTSD) using brain connectivity obtained from functional magnetic resonance imaging data. The classifier provided 90% accuracy. Brain connectivity features important for classification were generated for a pool of test subjects and permutation testing was used to identify significantly discriminative connections. Such heatmaps of significant paths were generated from 10 different interpretability algorithms based on variants of layer-wise relevance and gradient attribution methods. Since different interpretability algorithms make different assumptions about the data and model, their explanations had both commonalities and differences. Therefore, we developed a consensus across interpretability methods, which aligned well with the existing knowledge about brain alterations underlying PTSD. The confident identification of more than 20 regions, acknowledged for their relevance to PTSD in prior studies, was achieved with a voting score exceeding 8 and a family-wise correction threshold below 0.05. Our work illustrates how robustness and physiological plausibility of explanations can be achieved in interpreting classifications obtained from DNNs in diagnostic neuroimaging applications by evaluating convergence across methods. This will be crucial for trust in AI-based medical diagnostics in the future.

Accurate sleep stage classification is significant for sleep health assessment. In recent years, several machine-learning based sleep staging algorithms have been developed, and in particular, deep-learning based algorithms have achieved performance on par with human annotation. Despite improved performance, a limitation of most deep-learning based algorithms is their black-box behavior, which have limited their use in clinical settings. Here, we propose a cross-modal transformer, which is a transformer-based method for sleep stage classification. The proposed cross-modal transformer consists of a cross-modal transformer encoder architecture along with a multi-scale one-dimensional convolutional neural network for automatic representation learning. The performance of our method is on-par with the state-of-the-art methods and eliminates the black-box behavior of deep-learning models by utilizing the interpretability aspect of the attention modules. Furthermore, our method provides considerable reductions in the number of parameters and training time compared to the state-of-the-art methods. Our code is available at https://github.com/Jathurshan0330/Cross-Modal-Transformer. A demo of our work can be found at https://bit.ly/Cross_modal_transformer_demo.

SignificanceConvolutional neural networks (CNNs) show the potential for automated classification of different cancer lesions. However, their lack of interpretability and explainability makes CNNs less than understandable. Furthermore, CNNs may incorrectly concentrate on other areas surrounding the salient object, rather than the network's attention focusing directly on the object to be recognized, as the network has no incentive to focus solely on the correct subjects to be detected. This inhibits the reliability of CNNs, especially for biomedical applications.AimDevelop a deep learning training approach that could provide understandability to its predictions and directly guide the network to concentrate its attention and accurately delineate cancerous regions of the image.ApproachWe utilized Selvaraju et al.'s gradient-weighted class activation mapping to inject interpretability and explainability into CNNs. We adopted a two-stage training process with data augmentation techniques and Li et al.'s guided attention inference network (GAIN) to train images captured using our customized mobile oral screening devices. The GAIN architecture consists of three streams of network training: classification stream, attention mining stream, and bounding box stream. By adopting the GAIN training architecture, we jointly optimized the classification and segmentation accuracy of our CNN by treating these attention maps as reliable priors to develop attention maps with more complete and accurate segmentation.ResultsThe network's attention map will help us to actively understand what the network is focusing on and looking at during its decision-making process. The results also show that the proposed method could guide the trained neural network to highlight and focus its attention on the correct lesion areas in the images when making a decision, rather than focusing its attention on relevant yet incorrect regions.ConclusionsWe demonstrate the effectiveness of our approach for more interpretable and reliable oral potentially malignant lesion and malignant lesion classification.

Groundwater storage (GWS) decline, as well as total water storage (TWS) decline, in the (semi)arid Ordos Basin (China) poses great challenges to the water supply and ecological environment. In this study, the major factors causing the rapid loss of water storage during 2003–2020 are first investigated using correlation analysis, where the storage estimates are derived from the Gravity Recovery and Climate Experiment (GRACE) satellites. The major hydroclimatic drivers of monthly water storage changes are then explored among precipitation (P), evapotranspiration (ET), and runoff (RO) using an interpretable deep learning (IDL) method. The relative contribution of each driver is quantified by leveraging the interpretability nature of IDL. Results show that the GWS depletion (–0.82 cm/year) primarily accounts for the TWS loss (–0.73 cm/year) in the Ordos Basin under increased precipitation. The decreased TWS and GWS are both closely related to the increased vegetation density and coal production, indicating that they are the major drivers of the long-term water loss. At the monthly scale, the IDL method reveals that P and ET contribute over 75% to the changes of both TWS and GWS in most regions. The response lag of TWS to P and ET is generally 1–3 months. In contrast, GWS shows a more complicated response to P and ET with a longer lag range of 1–11 months in different regions due to the complicated Ordos Basin aquifer systems. These findings achieve a better understanding of hydrologic cycles and better guide sustainable water resources management in the Ordos Basin. [ABSTRACT FROM AUTHOR]

Neural Networks are one of the most popular methods nowadays given their high performance on diverse tasks, such as computer vision, anomaly detection, computer-aided disease detection and diagnosis or natural language processing. While neural networks are known for their high performance, they often suffer from the so-called “black-box” problem, which means that it is difficult to understand how the model makes decisions. We introduce a neural network topology based on Generalized Additive Models. By training an independent neural network to estimate the contribution of each feature to the output variable, we obtain a highly accurate and explainable deep learning model, providing a flexible framework for training Generalized Additive Neural Networks which does not impose any restriction on the neural network architecture. The proposed algorithm is evaluated through different simulation studies with synthetic datasets, as well as a real-world use case of Distributed Denial of Service cyberattack detection on an Industrial Control System. The results show that our proposal outperforms other GAM-based neural network implementations while providing higher interpretability, making it a promising approach for high-risk AI applications where transparency and accountability are crucial. [ABSTRACT FROM AUTHOR]

The widespread advancement of computer technology resulted in the increasing usage of deep learning models for predicting solar radiation. Numerous studies have been conducted to explore their research potential. Nevertheless, the application of deep learning models in optimizing building energy systems, particularly in a multi-step solar radiation prediction model for model predictive control (MPC), remains a challenging task. This is mainly due to the intricacy of the time series and the possibility of accumulating errors in multistep forecasts. In this study, we propose the development of a transformer-based attention model for predicting multi-step solar irradiation at least 24 h in advance. The model is trained and tested using measured solar irradiation data and temperature forecast data obtained from the Tokyo Meteorological Agency. The findings indicate that the transformer model has the capability to effectively mitigate the issue of error accumulation. Additionally, the generative model exhibits a significant improvement in accuracy, with a 62.35% increase when compared to the conventional regression LSTM model. Additionally, the transformer model has been shown to attain superior prediction stability, mitigate the effects of error accumulation in multi-step forecasting, and circumvent training challenges stemming from gradient propagation issues that can occur with recurrent neural networks. [ABSTRACT FROM AUTHOR]

Accurate sleep stage classification is significant for sleep health assessment. In recent years, several machine-learning based sleep staging algorithms have been developed, and in particular, deep-learning based algorithms have achieved performance on par with human annotation. Despite improved performance, a limitation of most deep-learning based algorithms is their black-box behavior, which have limited their use in clinical settings. Here, we propose a cross-modal transformer, which is a transformer-based method for sleep stage classification. The proposed cross-modal transformer consists of a cross-modal transformer encoder architecture along with a multi-scale one-dimensional convolutional neural network for automatic representation learning. The performance of our method is on-par with the state-of-the-art methods and eliminates the black-box behavior of deep-learning models by utilizing the interpretability aspect of the attention modules. Furthermore, our method provides considerable reductions in the number of parameters and training time compared to the state-of-the-art methods. Our code is available at https://github.com/Jathurshan0330/Cross-Modal-Transformer. A demo of our work can be found at https://bit.ly/Cross_modal_transformer_demo. [ABSTRACT FROM AUTHOR]

Abstract Background The significance of liver metastasis (LM) in increasing the risk of death for postoperative colorectal cancer (CRC) patients necessitates innovative approaches to predict LM. Aim Our study presents a novel and significant contribution by developing an interpretable fusion model that effectively integrates both free‐text medical record data and structured laboratory data to predict LM in postoperative CRC patients. Methods We used a robust dataset of 1463 patients and leveraged state‐of‐the‐art natural language processing (NLP) and machine learning techniques to construct a two‐layer fusion framework that demonstrates superior predictive performance compared to single modal models. Our innovative two‐tier algorithm fuses the results from different data modalities, achieving balanced prediction results on test data and significantly enhancing the predictive ability of the model. To increase interpretability, we employed Shapley additive explanations to elucidate the contributions of free‐text clinical data and structured clinical data to the final model. Furthermore, we translated our findings into practical clinical applications by creating a novel NLP score‐based nomogram using the top 13 valid predictors identified in our study. Results The proposed fusion models demonstrated superior predictive performance with an accuracy of 80.8%, precision of 80.3%, recall of 80.5%, and an F1 score of 80.8% in predicting LMs. Conclusion This fusion model represents a notable advancement in predicting LMs for postoperative CRC patients, offering the potential to enhance patient outcomes and support clinical decision‐making.

Parametric system identification, which is the process of uncovering the inherent dynamics of a system based on the model built with the observed inputs and outputs data, has been intensively studied in the past few decades. Recent years have seen a surge in the use of neural networks (NNs) in system identification, owing to their high approximation capability, less reliance on prior knowledge, and the growth of computational power. However, there is a lack of review on neural network modelling in the paradigm of parametric system identification, particularly in the time domain. This article discussed the connection in principle between conventional parametric models and three types of NNs including Feedforward Neural Networks, Recurrent Neural Networks and Encoder-Decoder. Then it reviewed the advantages and limitations of related research in addressing two major challenges of parametric system identification, including the model interpretability and modelling with nonstationary realisations. Finally, new challenges and future trends in neural network-based parametric system identification are presented in this article. [ABSTRACT FROM AUTHOR]

Most drugs entering clinical trials fail, often related to an incomplete understanding of the mechanisms governing drug response. Machine learning techniques hold immense promise for better drug response predictions, but most have not reached clinical practice due to their lack of interpretability and their focus on monotherapies. We address these challenges by developing DrugCell, an interpretable deep learning model of human cancer cells trained on the responses of 1,235 tumor cell lines to 684 drugs. Tumor genotypes induce states in cellular subsystems that are integrated with drug structure to predict response to therapy and, simultaneously, learn biological mechanisms underlying the drug response. DrugCell predictions are accurate in cell lines and also stratify clinical outcomes. Analysis of DrugCell mechanisms leads directly to the design of synergistic drug combinations, which we validate systematically by combinatorial CRISPR, drug-drug screening in vitro, and patient-derived xenografts. DrugCell provides a blueprint for constructing interpretable models for predictive medicine.

Background: The significance of liver metastasis (LM) in increasing the risk of death for postoperative colorectal cancer (CRC) patients necessitates innovative approaches to predict LM. Aim: Our study presents a novel and significant contribution by developing an interpretable fusion model that effectively integrates both free‐text medical record data and structured laboratory data to predict LM in postoperative CRC patients. Methods: We used a robust dataset of 1463 patients and leveraged state‐of‐the‐art natural language processing (NLP) and machine learning techniques to construct a two‐layer fusion framework that demonstrates superior predictive performance compared to single modal models. Our innovative two‐tier algorithm fuses the results from different data modalities, achieving balanced prediction results on test data and significantly enhancing the predictive ability of the model. To increase interpretability, we employed Shapley additive explanations to elucidate the contributions of free‐text clinical data and structured clinical data to the final model. Furthermore, we translated our findings into practical clinical applications by creating a novel NLP score‐based nomogram using the top 13 valid predictors identified in our study. Results: The proposed fusion models demonstrated superior predictive performance with an accuracy of 80.8%, precision of 80.3%, recall of 80.5%, and an F1 score of 80.8% in predicting LMs. Conclusion: This fusion model represents a notable advancement in predicting LMs for postoperative CRC patients, offering the potential to enhance patient outcomes and support clinical decision‐making. [ABSTRACT FROM AUTHOR]

Dissolved oxygen (DO) is one of the most important indicators of water quality and an essential measure for the aquatic organisms and the local ecosystem. DO concentrations in karst spring flow involves complex hydrological processes because of the heterogeneous nature of the karst system. In this study, an interpretable and explainable model that integrates the shapley additive explanations (SHAP) algorithm with the long short‐term memory network model (LSTM) is proposed to evaluate the contributions of karst spring discharge, precipitation, water temperature, and specific conductance to DO concentrations in karst spring flow. The hybrid model can predict the temporal fluctuations of DO levels and provide a robust characterization of DO behaviours. To demonstrate the applicability of the proposed model, we adopted the hydrological and meteorological data from Barton Springs, situated within a highly karstified segment of the Edwards aquifer characterized by extensive fractures and openings. The optimal prediction performance is achieved with a 14‐day time step, which is considered as the effective response time between various hydrological processes and DO concentrations at Barton Springs. It reveals that the influence of karst spring discharge, precipitation, water temperature, and specific conductance in previous 14 days collectively contribute to the current DO concentration in karst spring flow. The SHAP values of input features provide both local and global explanations, demonstrating the magnitude and the direction of each feature's impact on DO levels in karst spring flow. In descending order, the contributions of various hydrological processes to DO are ranked as follows: precipitation, karst spring discharge, temperature, and specific conductance. Precipitation and discharge exhibit positive SHAP values, indicating that increases in these hydrological processes contribute to higher DO levels in the karst flow. Water temperature and specific conductance have negative SHAP values, suggesting that higher water temperature and specific conductance will lead to decreased DO levels in the karst flow. [ABSTRACT FROM AUTHOR]

Reliable and accurate rice mapping using synthetic aperture radar (SAR) in cloudy and rainy areas is essential for achieving the United Nations Sustainable Development Goal 2 of 2030. An interpretable deep learning SAR rice area mapping method is proposed in this letter to suppress the interference of wetlands and other land covers to multitemporal SAR rice area mapping and improve the accuracy and confidence of the “black box” deep learning model results. Combining the temporal backscattering coefficients and interferometric coherence, three interpretable temporal features are extracted to effectively distinguish rice. Then, the explainable feature-aware network (XFANet), which can provide the learned importance weights of the normalization methods as self-interpretation, is constructed, and the pixel-wise gradient-weighted class activation mapping (PGCAM) post-hoc interpretation method is introduced to interpret the feature variation within the model. The experimental results in the Kampong Chhang and Kampong Chham provinces of Cambodia show that the proposed three interpretable features well suppressed the wetland disturbance to rice. With high interpretability, the overall accuracy (OA) of XFANet reaches 93.43%. [ABSTRACT FROM AUTHOR]

Visual explanation of “black-box” models allows researchers in explainable artificial intelligence (XAI) to interpret the model’s decisions in a human-understandable manner. In this letter, we propose interpretable class activation mapping for tree crown detection (Crown-CAM) that overcomes inaccurate localization and computational complexity of previous methods while generating reliable visual explanations for the challenging and dynamic problem of tree crown detection in aerial images. It consists of an unsupervised selection of activation maps, computation of local score maps, and noncontextual background suppression to efficiently provide fine-grain localization of tree crowns in scenarios with dense forest trees or scenes without tree crowns. In addition, two intersection over union (IoU)-based metrics are introduced to effectively quantify both the accuracy and inaccuracy of generated explanations with respect to regions with or even without tree crowns in the image. Empirical evaluations demonstrate that the proposed Crown-CAM outperforms the Score-CAM, Augmented Score-CAM, and Eigen-CAM methods by an average IoU margin of 8.7, 5.3, and 21.7 (and 3.3, 9.8, and 16.5), respectively, in improving the accuracy (and decreasing inaccuracy) of visual explanations on the challenging NEON tree crown dataset. [ABSTRACT FROM AUTHOR]

The recognition of an epitope by a T-cell receptor (TCR) is crucial for eliminating pathogens and establishing immunological memory. Prediction of the binding of any TCR–epitope pair is still a challenging task, especially for novel epitopes, because the underlying patterns are largely unknown to domain experts and machine learning models. To achieve a deeper understanding of TCR–epitope interactions, we have used interpretable deep learning techniques to gain insights into the performance of TCR–epitope binding machine learning models. We demonstrate how interpretable AI techniques can be linked to the three-dimensional structure of molecules to offer novel insights into the factors that determine TCR affinity on a molecular level. Additionally, our results show the importance of using interpretability techniques to verify the predictions of machine learning models for challenging molecular biology problems where small hard-to-detect problems can accumulate to inaccurate results.

Introduction: Existing large-scale preclinical cancer drug response databases provide us with a great opportunity to identify and predict potentially effective drugs to combat cancers. Deep learning models built on these databases have been developed and applied to tackle the cancer drug-response prediction task. Their prediction has been demonstrated to significantly outperform traditional machine learning methods. However, due to the “black box” characteristic, biologically faithful explanations are hardly derived from these deep learning models. Interpretable deep learning models that rely on visible neural networks (VNNs) have been proposed to provide biological justification for the predicted outcomes. However, their performance does not meet the expectation to be applied in clinical practice.Methods: In this paper, we develop an XMR model, an eXplainable Multimodal neural network for drug Response prediction. XMR is a new compact multimodal neural network consisting of two sub-networks: a visible neural network for learning genomic features and a graph neural network (GNN) for learning drugs’ structural features. Both sub-networks are integrated into a multimodal fusion layer to model the drug response for the given gene mutations and the drug’s molecular structures. Furthermore, a pruning approach is applied to provide better interpretations of the XMR model. We use five pathway hierarchies (cell cycle, DNA repair, diseases, signal transduction, and metabolism), which are obtained from the Reactome Pathway Database, as the architecture of VNN for our XMR model to predict drug responses of triple negative breast cancer.Results: We find that our model outperforms other state-of-the-art interpretable deep learning models in terms of predictive performance. In addition, our model can provide biological insights into explaining drug responses for triple-negative breast cancer.Discussion: Overall, combining both VNN and GNN in a multimodal fusion layer, XMR captures key genomic and molecular features and offers reasonable interpretability in biology, thereby better predicting drug responses in cancer patients. Our model would also benefit personalized cancer therapy in the future.

Deep learning algorithms (DLAs) are becoming hot tools in processing geochemical survey data for mineral exploration. However, it is difficult to understand their working mechanisms and decision-making behaviors, which may lead to unreliable results. The construction of a reliable and interpretable DLA has become a focus in data-driven geoscience discovery. This study utilized a SHapley Additive exPlanations (SHAP) framework, a popular post-hoc interpretability analysis method, incorporated with the variational autoencoder (VAE) to explore the contribution of geochemical elements for multivariate geochemical anomaly recognition. The sorting of element importance obtained by SHAP tool can provide a novel view for selecting a suitable elemental association related to mineralization. Based on the metallogenic model in the southeastern Hubei Province of China, a metallogenic-factor-based VAE model was constructed using an ad-hoc interpretable modeling technique. The interpretability of the model in identifying the abnormal distribution of the element associations can be improved by constructing a hidden layer and loss function containing metallogenic regularity and key metallogenic factors. The highly anomalous areas identified by the metallogenic-factor VAE model not only contain most of the known Au deposits, but also can reasonably identify the abnormal elemental associations related to ore-forming processes under the guidance of the metallogenic regularity. According to the output visualization of the new hidden layer, and the results of receiver operating characteristic curve and success-rate curve, the metallogenic-factor VAE model exhibits satisfied interpretability and performance. The geochemical anomalies identified in this study provide critical clues for future mineral exploration. [ABSTRACT FROM AUTHOR]

Structural health monitoring systems continuously monitor the operational state of structures, generating a large amount of monitoring data during the process. The structural responses of extreme events, such as earthquakes, ship collisions, or typhoons, could be captured and further analyzed. However, it is challenging to identify these extreme events due to the interference of faulty data. Real-world monitoring systems suffer from frequent misidentification and false alarms. Unfortunately, it is difficult to improve the system's built-in algorithms, especially the deep neural networks, partly because the current neural networks only output results and do not provide an interpretable decision-making basis. In this study, a deep learning-based method with visual interpretability is proposed to identify seismic data under sensor faults interference. The transfer learning technique is employed to learn the features of seismic data and faulty data with efficiency. A post hoc interpretation algorithm, termed Gradient-weighted Class Activation Mapping (Grad-CAM), is embedded into the neural networks to uncover the interest regions that support the output decision. The in situ seismic responses of a cable-stayed long-span bridge are used for method verification. The results show that the proposed method can effectively identify seismic data mixed with various types of faulty data while providing good interpretability. [ABSTRACT FROM AUTHOR]

Short-term residential load forecasting is of great significance to smart grid applications. Deep learning techniques, especially recurrent neural networks, can greatly improve the performance of prediction models. However, deep neural networks usually have low interpretability, which creates obstacles for customers to deeply understand the prediction results and make quick responses. In addition, the existing deep learning prediction methods rely heavily on the centralized training of massive data. However, the transmission of data from the client to the server poses a threat to the data security of customers. In this work, we propose an interpretable deep learning framework with federated learning for short-term residential load forecasting. Specifically, we propose a new automatic relevance determination network for feature interpretation, combined with the encoder–decoder architecture to achieve interpretable multi-step load prediction. In the edge computing network, the training scheme based on federated learning does not share the original data, which can effectively protect data privacy. The introduction of iterative federated clustering algorithm can alleviate the problem of non-independent and identical distribution of data in different households. We use two real-world datasets to verify the feasibility and performance of the proposed method. Finally, we discuss in detail the feature interpretation of these two datasets. [ABSTRACT FROM AUTHOR]