Search

In the contemporary data-driven era, the fields of machine learning, deep learning, big data, statistics, and data science are essential for forecasting outcomes and getting insights from data. This paper looks at how machine learning approaches can be used to anticipate solar power generation, assess heat exchanger heat transfer efficiency, and predict the thermo-physical properties of nanofluids. The review specifically focuses on the potential use of machine learning in solar thermal applications, perovskites, and photovoltaic power forecasting. Predictions of nanofluid characteristics and device performance may be more accurately made with the development of machine learning algorithms. The use of machine learning in the creation of new perovskites and the assessment of their effectiveness and stability is also included in the review. Additionally, the paper explores developments in artificial intelligence, particularly deep learning, in this area and offers insights into techniques for forecasting solar power, including PV production, cloud motion, and weather classification. [ABSTRACT FROM AUTHOR]

Purpose: The purpose of this paper is to develop surrogate models, using deep learning (DL), that can facilitate the application of EM analysis software. In the current status quo, electrical systems can be found in an ever-increasing range of products that are part of everyone's daily live. With the advances in technology, industries such as the automotive, communications and medical devices have been disrupted with new electrical and electronic systems. The innovation and development of such systems with increasing complexity over time has been supported by the increased use of electromagnetic (EM) analysis software. Such software enables engineers to virtually design, analyze and optimize EM systems without the need for building physical prototypes, thus helping to shorten the development cycles and consequently cut costs. Design/methodology/approach: The industry standard for simulating EM problems is using either the finite difference method or the finite element method (FEM). Optimization of the design process using such methods requires significant computational resources and time. With the emergence of artificial intelligence, along with specialized tools for automatic differentiation, the use of DL has become computationally much more efficient and cheaper. These advances in machine learning have ushered in a new era in EM simulations where engineers can compute results much faster while maintaining a certain level of accuracy. Findings: This paper proposed two different models that can compute the magnetic field distribution in EM systems. The first model is based on a recurrent neural network, which is trained through a data-driven supervised learning method. The second model is an extension to the first with the incorporation of additional physics-based information to the authors' model. Such a DL model, which is constrained by the laws of physics, is known as a physics-informed neural network. The solutions when compared with the ground truth, computed using FEM, show promising accuracy for the authors' DL models while reducing the computation time and resources required, as compared to previous implementations in the literature. Originality/value: The paper proposes a neural network architecture and is trained with two different learning methodologies, namely, supervised and physics-based. The working of the network along with the different learning methodologies is validated over several EM problems with varying levels of complexity. Furthermore, a comparative study is performed regarding performance accuracy and computational cost to establish the efficacy of different architectures and learning methodologies. [ABSTRACT FROM AUTHOR]

Data augmentation strategies have always been important in machine learning techniques and play a unique role in model performance optimization processes. Therefore, in recent years, these techniques have become popular in the artificial intelligence field. In this paper, a new data augmentation strategy is proposed based on the interpretation algorithm of deep convolutional neural networks, i.e., constructing new training samples by deeply exploiting key features extracted from interpretable networks to achieve sample augmentation. Thus, a novel supervised data augmentation approach known as Supervised Data Augmentation–Key Feature Extraction (SDA-KFE) was proposed. By introducing the Neural Network Interpreter-Segmentation Recognition and Interpretation (NNI-SRI) algorithm, an augmentation strategy is proposed that can balance the high accuracy and high robustness of the final model while ensuring a large amount of data augmentation. The advantages of the SDA-KFE algorithm are mainly reflected in the following aspects. First, it is easy to implement. This algorithm is implemented based on the lightweight NNI-SRI algorithm, which lays the foundation for the implementation of SDA-KFE so that it can be easily implemented on convolutional neural networks. Second, this model, which is widely applicable, can be applied to almost any deep convolutional network. Through research and experiments on this proposed algorithm, SDA-KFE can be applied in graphical image binary classification and multiclassification models. Third, SDA-KFE can rapidly construct data samples with diverse variations. Under the premise of determining the classification labels of the generated samples, the distribution of the feature unit composition of the samples can be controlled. Compared with traditional data augmentation methods, SDA-KFE can control the direction of the model performance, i.e., the balance between the pursuit of high accuracy and robust performance of the model. Therefore, the novel supervised augmentation approach proposed in this paper is relevant for optimizing deep convolutional neural networks, solving model overfitting, augmenting data types, etc. The data augmentation algorithm proposed in this paper can be regarded as a useful supplement to traditional data augmentation methods, such as horizontal or vertical image flipping, cropping, color transformation, extension and rotation. [ABSTRACT FROM AUTHOR]

Explainable Artificial Intelligence (XAI) is targeted at understanding how models perform feature selection and derive their classification decisions. This paper explores post-hoc explanations for deep neural networks in the audio domain. Notably, we present a novel Open Source audio dataset consisting of 30,000 audio samples of English spoken digits which we use for classification tasks on spoken digits and speakers' biological sex. We use the popular XAI technique Layer-wise Relevance Propagation LRP to identify relevant features for two neural network architectures that process either waveform or spectrogram representations of the data. Based on the relevance scores obtained from LRP, hypotheses about the neural networks' feature selection are derived and subsequently tested through systematic manipulations of the input data. Further, we take a step beyond visual explanations and introduce audible heatmaps. We demonstrate the superior interpretability of audible explanations over visual ones in a human user study. • We present a novel audio dataset consisting of 30,000 audio samples of spoken digits. • We use LRP to explain predictions of two different models in the audio domain. • We confirm hypotheses about the neural networks' use of features from explanations. • We present audible explanations and demonstrate their superior interpretability. [ABSTRACT FROM AUTHOR]

Multimodal emotion recognition, that is, emotion recognition uses machine learning to generate multi-modal features on the basis of videos which has become a research hotspot in the field of artificial intelligence. Traditional multi-modal emotion recognition method only simply connects multiple modalities, and the interactive utilization rate of modal information is low, and it cannot reflect the real emotion under the conflict of modal features well. This article first proves that effective weighting can improve the discrimination between modalities. Therefore, this paper takes into account the importance differences between multiple modalities, and assigns weights to them through the importance attention network. At the same time, considering that there is a certain complementary relationship between the modalities, this paper constructs an attention network with complementary modalities. Finally, the reconstructed features are fused to obtain a multi-modal feature with good interaction. The method proposed in this paper is compared with traditional methods in public datasets. The test results show that our method is accurate in It performs well in both the rate and confusion matrix metrics. [ABSTRACT FROM AUTHOR]

Nowadays, Deep Learning is drastically revolutionizing financial research as well as industry. Many methods have been discussed in the last few years, mainly related to option pricing. In fact, traditional approaches such as Monte Carlo simulation or finite difference methods are seriously harmed by multi-dimensional underlying and path dependency. Thus, dealing with particular contracts such as American multi-asset options is still rough. This paper addresses such a problem by pricing said put options with a novel meshless methodology, named Physics-Informed Neural Networks (PINNs), based on Artificial Intelligence. PINN paradigm has been recently introduced in Deep Learning literature. It exploits the theoretical background of the universal approximation theorem for neural networks to solve Partial Differential Equations numerically. This Deep Learning meshless method incorporates the equation and its initial and boundary conditions thanks to a specially designed loss function. We develop a suitable PINN for the proposed problem by introducing an algorithmic trick for improving the convergence of the free boundary problem. Furthermore, the worthiness of the proposal is assessed by several experiments concerned with single and multi-asset options. Finally, a parametric model is built to benefit further studies of option value behaviour related to particular market conditions. [ABSTRACT FROM AUTHOR]

• The paper presents a lightweight model with satisfactory performance in complex industrial noise environments. • Data fusion using data from multiple sensors for more complete fault information. • Design a unique grayscale feature map based on IMF components to obtain multi-sensor and multi-frequency band fault features. • Design a UL-GoogLeNet lightweight fault diagnosis model based on GoogLeNet and ULSAM. As artificial intelligence and sensor technology develop rapidly, intelligent fault diagnosis methods based on deep learning are widely used in industrial production. However, in practical industrial applications, the complex noise environment affects the performance of the diagnostic model, and the huge model parameters cannot meet the requirements of low cost and high performance in industrial production. To address the above problems, this paper proposes a lightweight intelligent fault diagnosis model using multi-sensor data fusion that not only meets the lightweight requirements of "small, light, and fast", but also realizes high accuracy diagnosis in noisy environments. Firstly, the vibration signals from different sensors of rolling bearings are processed using the variational mode decomposition (VMD) to design a unique method of constructing grayscale feature maps based on each intrinsic modal function (IMF) component. Then, the ultra-lightweight GoogLeNet model (UL-GoogLeNet) is constructed to adjust the traditional GoogLeNet structure, while the Ultra-lightweight subspace attention module (ULSAM) is introduced to reduce the model parameters and enhance the feature extraction capability. UL-GoogLeNet is trained and tested by dividing the grayscale feature maps into training and testing sets to realize the intelligent recognition of different fault types in rolling bearings. Experiments are conducted on two datasets and compared with multiple methods, and the final experimental results prove the effectiveness and superiority of the proposed method in this paper. [ABSTRACT FROM AUTHOR]

Progress in the field of Non-Intrusive Load Monitoring (NILM) has been attributed to the rise in the application of artificial intelligence. Nevertheless, the ability of energy disaggregation algorithms to disaggregate different appliance signatures from aggregated smart grid data poses some privacy issues. This paper introduces a new notion of disclosure risk termed energy disaggregation risk. The performance of Sequence-to-Sequence (Seq2Seq) NILM deep learning algorithm along with three activation extraction methods are studied using two publicly available datasets. To understand the extent of disclosure, we study three inference attacks on aggregated data. The results show that Variance Sensitive Thresholding (VST) event detection method outperformed the other two methods in revealing households' lifestyles based on the signature of the appliances. To reduce energy disaggregation risk, we investigate the performance of two privacy-preserving mechanisms based on microaggregation and Discrete Fourier Transform (DFT). Empirically, for the first scenario of inference attack on UK-DALE, VST produces disaggregation risks of 99%, 100%, 89% and 99% for fridge, dish washer, microwave, and kettle respectively. For washing machine, Activation Time Extraction (ATE) method produces a disaggregation risk of 87%. We obtain similar results for other inference attack scenarios and the risk reduces using the two privacy-protection mechanisms. [ABSTRACT FROM AUTHOR]

Technology has grown rapidly in recent years, and new solutions that rely on Machine Learning (ML) and Artificial Intelligence (AI) are introduced every day. With such fast-paced advancement, inspecting and fully comprehending how given models make decisions is becoming problematic. The complex decision-making process of these models has become a black box, making it challenging to unravel how they work; therefore, eXplainable Artificial Intelligence (XAI) methods are crucial for further development. This paper discusses how state-of-the-art techniques determine classifications and why they need to be revised to understand the prediction-generating process fully. It compares those existing solutions with the new method called Principal Image Sections Mapping - PRISM, which relies on Principal Component Analysis and allows visualising the most significant features recognised by a given Convolutional Neural Network. PRISM is implemented in a piece of software called TorchPRISM that can generate and present the clustering based on the method's output. The result can indicate ambiguous classes discrimination; thus, the possibility of automating the output analysis process is also discussed. The paper's main objective is to examine how PRISM enhances the current understanding of the decision-making process and introduce a tool that can facilitate analysing the output. PRISM implementation (TorchPRISM) can be found in the public GitHub repository: https://github.com/szandala/TorchPRISM • PRISM method can explain CNN's decisions in an understandable way for various users. • PRISM uses PCA to point features and highlights them using an RGB-coloured image mask. • PRISM's effectiveness is assessed based on qualitative and quantitative measures. • TorchPRISM software allows the identification of ambiguous classes in large datasets. • PRISM overcomes the deficiencies of existing eXplainable AI and saliency map methods. [ABSTRACT FROM AUTHOR]

Differential privacy (DP), as a promising privacy-preserving model, has attracted great interest from researchers in recent years. At present, research on the combination of deep learning and DP is active. In contrast, another widely used artificial intelligence technique, swarm intelligence (SI), has received little attention in the context of DP even though it also triggers privacy concerns. For this reason, this paper attempts to combine DP and SI for the first time and proposes a general differentially private swarm intelligence algorithm framework (DPSIAF). By utilizing the exponential mechanism, this framework can easily develop existing SI algorithms into the private versions. As examples, we apply it to four popular SI algorithms, and the related analyses demonstrate the effectiveness of our DPSIAF. More importantly, our private algorithms also exhibit some interesting experimental phenomena. In some cases, their performance is not strictly affected by the privacy budget, and one of them even outperforms its non-private version. These findings are different from the conventional cognition, which indicates DP owns the potential to serve as an optimization tool for some SI algorithms. Our study may provide a new perspective on DP and promote the synergy between metaheuristic optimization community and privacy computing community. [ABSTRACT FROM AUTHOR]

Although deep learning models can solve complex prediction problems, they have been criticized for being 'black boxes'. This implies that their decisions are difficult, if not impossible, to explain by simply inspecting their internal knowledge structures. Explainable Artificial Intelligence has attempted to open the black-box through model-specific and agnostic post-hoc methods that generate visualizations or derive associations between the problem features and the model predictions. This paper proposes a new method, termed REPROT, that explains the decisions of complex deep learning architectures based on local reducts of an image. A 'reduct' is a set of sufficiently descriptive features that can fully characterize the acquired knowledge. The created reducts are used to build a 'prototype image' that visually explains the inference obtained by a black-box model for an image. We focus on deep learning architectures whose complexity and internal particularities demand adapting existing model-specific explanation methods, making the explanation process more difficult. Experimental results show that the black-box model can detect an object using the prototype image generated from the reduct. Hence, the explanations will be given by "the minimum set of features sufficient for the neural model to detect an object". The confidence scores obtained by architectures such as Inception, Yolo, and Mask R-CNN are higher for prototype images built from the reduct than those built from the most important superpixels according to the LIME method. Moreover, the target object is not detected on several occasions through the LIME output, thus supporting the superiority of the proposed explanation method. [ABSTRACT FROM AUTHOR]

Time series are a kind of streaming data, which are chaotic and sequential. As real-world time series data are often not available at once and drift with time growth, Incremental Learning (IL) is well suited for Time Series Prediction (TSP). Most previous incremental TSP algorithms are limited by the assumption of data balance. However, real-world time series data are often unbalanced, with long-tailed distribution and other characteristics resulting in the failure of IL algorithms. In this paper, a balanced-driven Active Learning (AL) strategy is proposed to deal with data imbalance problems in IL processes. What's more, by integrating the advantages of Deep Learning (DL) and the Broad Learning System (BLS), a novel Deep-Broad Learning (DeepBL) network with its incremental learning algorithm is proposed. The proposed A ctive Learning-based I ncremental Deep - B road L earning (AI_DeepBL) algorithm is applied to real-world univariate and multivariate time series datasets and achieves superior performance compared with classical and state-of-the-art TSP algorithms. [ABSTRACT FROM AUTHOR]

Although the field of eXplainable Artificial Intelligence (XAI) has a significant interest these days, its implementation within cyber security applications still needs further investigation to understand its effectiveness in discovering attack surfaces and vectors. In cyber defence, especially anomaly-based Intrusion Detection Systems (IDS), the emerging applications of machine/deep learning models require the interpretation of the models' architecture and the explanation of models' prediction to examine how cyberattacks would occur. This paper proposes a novel explainable intrusion detection framework in the Internet of Things (IoT) networks. We have developed an IDS using a Short-Term Long Memory (LSTM) model to identify cyberattacks and explain the model's decisions. This uses a novel set of input features extracted by a novel SPIP (S: Shapley Additive exPlanations, P: Permutation Feature Importance, I: Individual Conditional Expectation, P: Partial Dependence Plot) framework to train and evaluate the LSTM model. The framework was validated using the NSL-KDD, UNSW-NB15 and TON_IoT datasets. The SPIP framework achieved high detection accuracy, processing time, and high interpretability of data features and model outputs compared with other peer techniques. The proposed framework has the potential to assist administrators and decision-makers in understanding complex attack behaviour. • We propose a novel explainable deep learning-based intrusion detection method that provides global and local explanations. • We use input features extracted by the proposed framework to train and evaluate the proposed intrusion detection method. • We demonstrate the proposed framework's ability to effectively enhance the interpretability of cyber defence systems in IoT networks. [ABSTRACT FROM AUTHOR]

Recent developments in the field of deep learning have led to the widespread integration of artificial neural networks in various domains of application. Prominent contemporary artificial neural network training techniques are based on first-order gradient computation. The emphasis on algorithmic performance has driven the emergence of variant artificial neural network training methodologies. Estimation theory, traditionally considered a sub-field of statistics and signal processing, has been explored by various researchers for the development of non-gradient based training methods. Articles published with the aim of utilizing estimation-based artificial neural network training techniques have shown promising results. We identify the integration of estimation theory within the artificial neural network training procedure as intelligent estimation. In this paper, the field of intelligent estimation is analyzed in greater depth with emphasis on the algorithmic performance of novel implementations. Intelligent estimation with applications in the professional domain is also considered, and will help lay the foundation for future research in the literature. [ABSTRACT FROM AUTHOR]

The field of digital histopathology has seen incredible growth in recent years. Digital pathology is becoming a relevant tool in healthcare, industrial and research sectors to reduce the saturation of pathology departments and improve the productivity of pathologists by increasing diagnostic accuracy and reducing turnaround times. Artificial Intelligence (AI) algorithms may be used for the identification of relevant regions, extraction of features from a histological image and overall classification of images into specific classes. The combination of digital histopathology imaging and AI therefore presents a significant opportunity for the support of the pathologists' tasks and opens up a whole new world of computational analysis. In this paper, we have analysed the present, the challenges and the future of the computational pathology discussing the different existing strategies to overcome its main limitations and ensure the computational pathology acceptance. The lack of labelled data, which is the possibly largest challenge for all medical AI applications, is even more pronounced in computational pathology because of the multi-gigapixel nature of the images and high data heterogeneity. We consider the future of the computational pathology is the combination of weak label strategies with active learning and crowdsourcing scenarios since it would remove some of the workload from clinical experts and manual annotation obtaining clinically satisfactory performance with minimal annotation effort. In addition, we believe areas such as explainable AI, data fusion and secure role-based data sharing will be receiving increasing research attention in computational pathology in the close future. • Computational pathology is the automatic analysis of histological images. • The present, challenges and future of computational pathology are analyzed. • Challenges: multi-gigapixel images, data heterogeneity and lack of labelled data. • Future directions: explainable AI, data fusion and secure role-based data sharing. • Combining weak label strategies, active learning and crowdsourcing is promising. [ABSTRACT FROM AUTHOR]

In recent years, elliptic partial differential equations (PDEs) have been challenging in numerical mathematics, engineering, and physics. In this paper, we proposed a novel approach called Multilayer Neural Network for Partial Differential Equations (MLNPDE) based on deep learning algorithms by exploring the elliptic PDE family under certain conditions such as initial conditions and boundary conditions to govern approximation solutions of PDEs. In the proposed model, we include a multilayer neural network using a densely connected network. Moreover, the approximation solution underlying PDEs can be expressed in two terms: the first term satisfies the boundary conditions, and the second term is a function of the unknown parameters that were estimated by our proposed model. Furthermore, we derive a normal form of the boundary conditions on the first term of the approximation solution for two-dimensional elliptic partial differential equation problems with an arbitrary domain. The numerical results show that our method achieved tremendous state-of-the-art performance in accuracy and efficiency compared to the classical methods. [ABSTRACT FROM AUTHOR]