Search

In the subjective evaluation process, the hesitant fuzzy set (HFS), as a convenient and robust presentation tool, cannot only suitably address the decision makers' (DMs') or experts' hesitant and uncertain issues but also can arise the dimension curse puzzle. Furthermore, the decision-making result is just derived according to the given objective and subjective information, without considering the DM's subjective evaluation and the environment's dynamic influence. Unlike the previous studies, this paper tries to address them from the deep learning viewpoint. To this end, we first define the non-equidimensional HFS (NHFS) and then introduce the equidimensional and classification characters into the NHFS to further develop the equidimensional HFS (EHFS) and the EHFS with the optimal classification result. Then, the equidimensional hesitant fuzzy-generative adversarial network (EHF-GAN) model is proposed to transform the hesitant fuzzy information from the non-equidimensional to the equidimensional form. The generalization and the convergence of the new model are proven to show the models' reasonability. In addition, the double-learning algorithm of the EHF-GAN model is designed, which can fuse the DMs' dynamic judgments and derive the optimal decision-making results. Lastly, this paper applies the proposed model and algorithm to an illustrative example of the new smart city enterprises and then shows their feasibility and effectiveness. [ABSTRACT FROM AUTHOR]

As deep learning models become increasingly integral to various 3D applications, concerns about their vulnerability to adversarial attacks grow in tandem. This paper addresses the challenge of enhancing the transferability of 3D adversarial attacks, a critical aspect for evaluating model robustness across diverse scenarios. We propose a novel approach leveraging scale and shear transformations to generate adversarial examples that exhibit improved transferability across multiple 3D models. Our methodology involves carefully integrating scale and shear transformations into the adversarial perturbation generation process with only a marginal increase in computational time. The proposed attack method operates within the Carlini-Wagner (CW) optimization framework. For each iteration, it employs two hyperparameters: p a , determining the probability of transforming the input point cloud, and p s , deciding whether to shear or scale the point cloud. Limited to scaling and shearing transformations, Scale and Shear (SS) attack seamlessly integrates with established attack methods, enhancing flexibility and compatibility in adversarial attacks on 3D models. Extensive experiments show that the SS attack proposed in this paper can be seamlessly combined with the existing state-of-the-art (SOTA) 3D point cloud attack methods to form more powerful attack methods, and the SS attack improves the transferability over 3.6 times compared to the baseline. Moreover, while substantially outperforming the baseline methods, the SS attack achieves SOTA transferability under various defenses. Our code will be available online at: https://github.com/cuge1995/SS-attack. [ABSTRACT FROM AUTHOR]

Human behavior may be one of the most challenging phenomena to model and validate. This paper proposes a method for automatically extracting and compiling evidence on human behavior determinants into a knowledge graph. The method (1) extracts associations of behavior determinants and choice options in relation to study groups and moderators from published studies using Natural Language Processing and Deep Learning, (2) synthesizes the extracted evidence into a knowledge graph, and (3) sub-selects the model components and relationships that are relevant and robust. The method can be used to either (4a) construct a structurally valid simulation model before proceeding with calibration or (4b) to validate the structure of existing simulation models. To demonstrate the feasibility of the method, we discuss an example implementation with mode of transport as behavior choice. We find that including non-frequently studied significant behavior determinants drastically improves the model's explanatory power in comparison to only including frequently studied variables. The paper serves as a proof-of-concept which can be reused, extended or adapted for various purposes. • The structure of behavior models should be validated using existing evidence. • Deep Learning can help automatize behavior evidence extraction and synthesis. • Infrequently studied significant variables improve model performance drastically. [ABSTRACT FROM AUTHOR]

Adversarial Training has proved to be an effective training paradigm to enforce robustness against adversarial examples in modern neural network architectures. Despite many efforts, explanations of the foundational principles underpinning the effectiveness of Adversarial Training are limited and far from being widely accepted by the Deep Learning community. Moreover, very few research works investigated the limitations of robust Convolutional Neural Networks beyond the well-known accuracy drop on natural images. In this paper, we describe surprising properties of these models, shedding light on mechanisms through which robustness against adversarial attacks is implemented. We also highlight limitations and failure modes that were not discussed in prior works. Through extensive analyses on a wide range of architectures and datasets, we empirically demonstrate that adversarially-trained Convolutional Neural Networks do not exploit efficiently the model capacity and that the simplicity biases induced by Adversarial Training may lead to undesired behaviors. [ABSTRACT FROM AUTHOR]

Sensor nodes (industrial nodes) clustering is an efficacious method for gaining local working conditions real-timely in the production process. The existing deep graph neural networks-based industrial nodes clustering studies construct static graphs to describe the potential spatial correlation among industrial nodes, but the temporal correlation among industrial nodes is ignored, resulting in inferior real-time performance. Meanwhile, these deep learning models fail to obtain good clustering results due to the large-scale and noise-containing characteristics of industrial nodes. In this paper, a large-scale graph clustering method (LsGCM) is proposed for cell conditions spatio-temporal localization in aluminum electrolysis to address the above issues. Specifically, a spatio-temporal graph construction method based on industrial process mechanism knowledge and local working condition boundary types statistical feature fusion is proposed for capturing effective spatio-temporal correlations between industrial nodes. Then, hysteresis knowledge of industrial production system is used to guide the nodes merging, thereby avoiding the incorrect merging of time-asynchronous nodes when performing the spatio-temporal region detection of local working conditions. Finally, combined with the actual spatio-temporal location information of industrial nodes, a novel spatio-temporal information expansion method (StIEM) of the sub-spatio-temporal graph describing local working condition is proposed for obtaining spatio-temporal feature information of the neighborhood, which is beneficial to improve the performance of industrial node clustering. Validation studies on a numerical simulation example and multiple real aluminum electrolysis industrial datasets certificate the effectiveness and superiority of the proposed method. In particular, compared with existing graph clustering methods, our method improves the clustering accuracy by 7.79%. • A large-scale graph clustering method for cell conditions spatio-temporal localization in aluminum electrolysis is proposed. • A spatio-temporal graph adaptive sparsization method based on mechanism knowledge and statistical feature fusion is proposed. • A spatio-temporal information expansion method based on the actual spatio-temporal location of nodes is proposed. [ABSTRACT FROM AUTHOR]

The classification of hazard is critical in industrial informatics, as it enhances early safety alerts, supports decision-making, and facilitates policy assessment. However, previous studies have generally neglected the temporal attributes of hazards, thereby constraining the effectiveness of models. This paper introduces a new model for hazard classification termed DLGM. DLGM represents a deep learning framework, with the structural parameters of grey models to encapsulate the hazard temporal attributes. To better accommodate the fluctuations of hazard series, a new grey model termed FSGM(1,1) equipped with Fourier series is proposed. Moreover, DLGM leverages a novel hierarchical feature fusion neural network (HFFNN) to optimize feature processing. Extensive experiments across three hazard themes involving 18 large-scale industrial processes have demonstrated the competitiveness of DLGM (e.g., it surpasses benchmark models such as Random Forest and BERT by approximately 2% in both accuracy and F1), the suitability of FSGM(1,1) (e.g., its mean absolute percentage error is less than 10% and mean squared error is below 0.02) and the effectiveness of HFFNN (e.g., it enhances the accuracy and F1 of DLGM by about 1%). [ABSTRACT FROM AUTHOR]

Fire accidents continually threaten lives and property, posing a great risk to public safety. Therefore, timely, accurate fire smoke and flame detection technology provides a technical foundation for ensuring personnel and property safety. This paper proposes a deep transfer learning model that integrates attention mechanisms and pruning techniques into the DenseNet network (P-DenseNet-A-TL) for detecting fire smoke and flame targets. To reduce the computational complexity of the fire detection model, we simplified the DenseNet network structure. To improve the recognition accuracy of the fire detection model, we incorporated attention mechanisms including a channel attention module (CAM) and spatial attention module (SAM) into the pruned dense network structure. To expedite the model training process, we also introduced a transfer learning model. Furthermore, to enhance the generalization ability and robustness of the model, we created a large-scale fire smoke and flame dataset. The experimental results show that our model (P-DenseNet-A-TL) achieved a test accuracy of 99.06%, F1 score of 99.09%, area under the curve (AUC) of 0.97, and a detection speed of 756 frames per second (FPS). The comparison experimental results and ablation experimental results indicate that our method achieves high detection accuracy and efficiency. Additionally, it possesses strong generalization capability and robustness. [ABSTRACT FROM AUTHOR]

As the key step of anomaly detection for multivariate time-series (MTS) data, learning the relations among different variables has been explored by many approaches. However, most existing approaches overlook the heterogeneity among variables, that is, different types of variables (continuous numerical variables, discrete categorical variables or hybrid variables) may have different edge distributions. In this paper, we propose a novel semi-supervised anomaly detection framework based on a heterogeneous feature network (HFN) for MTS. Specifically, we first combine the embedding similarity subgraph generated by sensor embedding and the feature value similarity subgraph generated by sensor values to construct a time-series heterogeneous graph, which fully utilizes the rich heterogeneous mutual information among variables. Then, a prediction model containing nodes and channel attentions is jointly optimized to obtain better time-series representations. This approach fuses the state-of-the-art technologies of heterogeneous graph structure learning (HGSL) and representation learning. Experimental results on four sensor datasets from real-world applications demonstrate that our approach achieves more accurate anomaly detection compared to baseline methods, laying a foundation for the rapid positioning of anomalies. [ABSTRACT FROM AUTHOR]

With the advance of deep learning, adversarial attack and defense has becoming a hot research topic. However, existing defense methods rely on the prior knowledge of the adversarial attacks, and are also vulnerable to adaptive attacks. In this paper, towards a secure automatic speaker verification system, a novel attack-agnostic defense method is proposed, named by SampleShield , which consists of a pairwise random downsampling (PRD) module and an upsampling module achieved by speech super-resolution (SSR). The PRD module can destroy the adversarial perturbations by downsampling, which also introduces randomness and non-differentiability, leading to its resistance to adaptive attacks. PRD does not use any adversarial example as training data, which makes it an attack-agnostic defense method with good generalization ability. Furthermore, the upsampling module achieved by a neural network can recover the downsampled speech to its original quality. In summary, given the experimental results and analysis on the benchmark public datasets, by fusing the downsampling and upsampling modules, SampleShield achieves excellent performance. On adversarial defense, SampleShield obtains an error rate about 5.2% which is quite close to the idea lower bound value 3.2%. On speech quality, SampleShield gets a 5 dB signal-to-noise ratio improvement with respect to the best existing method. • An adversarial defense framework by fusing downsampling and upsampling. • Pairwise random downsampling to resist normal and adaptive adversarial attacks. • A new deep learning architecture to perform speech super-resolution. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of approximating functions from a Korobov space on [ − 1 , 1 ] d by ReLU shallow neural networks and present a rate O (m − 2 5 (1 + 2 d) log ⁡ m) of uniform approximation by networks of m hidden neurons. This is achieved by combining a novel Fourier analysis approach and a probability argument. We apply our approximation theory to a learning algorithm for regression based on ReLU shallow neural networks and derive learning rates of order O (N − 4 (d + 2) 9 d + 8 log ⁡ N) for the excess generalization error with the sample size N when the regression function lies in the Korobov space. [ABSTRACT FROM AUTHOR]

Explainability is crucial in domains where system decisions have significant implications for human trust in black-box models. Lack of understanding regarding how these decisions are made hinders the adoption of so-called clinical decision support systems. While neural networks and deep learning methods exhibit impressive performance, they remain less explainable than white-box approaches. Artificial Hydrocarbon Networks (AHN) is an effective black-box model that can be used to support critical clinical decisions if accompanied by explainability mechanisms to instill confidence among clinicians. In this paper, we present a use case involving global and local explanations for AHN models, provided with an automatic procedure so-called eXplainable Artificial Hydrocarbon Networks (XAHN). We apply XAHN to preeclampsia prognosis, enabling interpretability within an accurate black-box model. Our approach involves training a suitable AHN model using the cross-validation with ten repetitions, followed by a comparative analysis against four well-known machine learning techniques. Notably, the AHN model outperformed the others, achieving an F1-score of 74.91%. Additionally, we assess the efficacy of our XAHN explainer through a survey applied to clinicians, evaluating the goodness and satisfaction of the provided explanations. To the best of our knowledge, this work represents one of the earliest attempts to address the explainability challenge in preeclampsia prediction. [ABSTRACT FROM AUTHOR]

• Equity value and debt maturity make this portfolio more complete by using KMV model. • The objectives of the proposed portfolio include order moments and default distance. • Improved NSGA-III with chaos map is developed to solve five-objective optimization. • Static penalty is transformed the constrained optimization to unconstrained problem. • CNN-BiLSTM is firstly used to predict trend when selecting stocks. Portfolio management is a critical aspect of investment strategies, with the goal to balance the low-risk and high-return investments. Despite this, existing portfolios frequently overlook the integration of stock selection outcomes and underutilize data from listed companies, leading to suboptimal portfolio performance. Addressing these shortcomings, this paper introduces a hybrid system involving stock selection and portfolio optimization. In stock selection, the system employs a combination of convolutional neural network and bi-directional recurrent neural network to predict stock trends. This approach enables the identification of stocks likely to appreciate in value, setting the stage for their inclusion in the subsequent optimization process. For portfolio optimization, the study formulates a five-objective optimization problem that incorporates mean, variance, skewness, kurtosis, and distance-to-default as key considerations. To solve the many-objective constrained optimization problem, an advanced strategy employing a static penalty function and an improved Non-dominated Sorting Genetic Algorithm III (NSGA-III) based on tent chaotic mapping is utilized. The efficacy of the proposed hybrid system is rigorously tested through three sets of ablation experiments alongside two discussions focused on its robustness and computational efficiency. The findings from these investigations reveal that the hybrid system outperforms traditional approaches, reducing risks and improving returns for investors. [ABSTRACT FROM AUTHOR]

The k Nearest Neighbors (k NN) search in high-dimensional space is a fundamental problem with various applications. In this paper, we try to solve this problem by using deep neural networks (DNNs). We apply DNNs to represent complex correlations between high-dimensional objects, enabling us to project similar objects into the same class, thus reducing the search cost. Based on DNNs, we propose two novel techniques to improve query efficiency while keeping accuracy. First, traditional DNNs typically demand extensive training data for achieving high accuracy. To decrease the training size, we design a multi-module DNN framework comprising several small modules. Each module learns to capture part of knowledge for the given query. The collective output of these sub-modules is then seamlessly integrated to form the final result. Second, with machine learning models, the size of candidates are unbounded. Thus, we design a linear-time data layout refinement algorithm, aiming to limit the number of candidates to a small constant. Empirically we find that our approach significantly outperforms the state-of-the-art methods in terms of both time efficiency and space efficiency while still attaining comparable or better accuracy. [ABSTRACT FROM AUTHOR]

This paper presents a comprehensive empirical investigation into the interactions between various randomization techniques in Deep Neural Networks (DNNs) and their impact on learning performance. It is well-established that injecting randomness into the training process of DNNs, through various approaches, at different stages, is often beneficial for reducing overfitting and improving generalization. Nonetheless, the interactions between randomness techniques such as weight noise, dropout, and many others remain poorly understood. Consequently, it is challenging to determine which methods can be effectively combined to optimize DNN performance. To address this issue, we categorize the existing randomness techniques into four key types: injection of noise/randomness at the data, model structure, optimization or learning stage. We use this classification to identify gaps in the current coverage of potential mechanisms for the introduction of randomness, leading to proposing two new techniques: adding noise to the loss function and random masking of the gradient updates. In our empirical study, we employ a Particle Swarm Optimizer (PSO) for hyperparameter optimization (HPO) to explore the space of possible configurations to determine where and how much randomness should be injected to maximize DNN performance. We assess the impact of various types and levels of randomness for DNN architectures across standard computer vision benchmarks: MNIST, FASHION-MNIST, CIFAR10, and CIFAR100. Across more than 30 000 evaluated configurations, we perform a detailed examination of the interactions between randomness techniques and their combined impact on DNN performance. Our findings reveal that randomness through data augmentation and in weight initialization are the main contributors to performance improvement. Additionally, correlation analysis demonstrates that different optimizers, such as Adam and Gradient Descent with Momentum, prefer distinct types of randomization during the training process. A GitHub repository with the complete implementation and generated dataset is available.2 [ABSTRACT FROM AUTHOR]

This paper introduces Sill-Rgan, a novel Generative Adversarial Network (GAN) designed to improve hyperspectral image (HSI) classification under varying lighting conditions. Sill-Rgan uniquely maps different light condition domains, enhancing sample classification robustness and generating new virtual samples. Addressing challenges like high spectral dimensionality and noise in HSI classification, our approach utilizes a deep proxy-based learning framework. It integrates and improves advanced GAN models and multitask networks for optimal training stability and loss function optimization. The model's mapping network is adept at generating domain-specific latent codes, enabling the transformation of original hyperspectral data into enhanced versions. Extensive experiments conducted on hyperspectral datasets of agricultural products under diverse indoor and outdoor lighting conditions confirm the effectiveness of Sill-Rgan. The results highlight the model's adaptability in both supervised and semi-supervised learning scenarios, yielding exceptional classification accuracy and enhanced data quality. The versatile potential of Sill-Rgan extends its applicability to a broad range of spectral data classifications, underlining its significant contribution to hyperspectral imaging. This advancement opens new avenues in machine vision systems, particularly in scenarios with dynamic lighting challenges. [ABSTRACT FROM AUTHOR]

• The proposed FE-RNN learns incrementally and inferences on unseen time-series data. • The FE-RNN is assessed on chaotic time-series datasets for its forecasting ability. • An embedded fuzzy network provides interpretability of the neural network structure. • The FE-RNN based GA-fMACDH strategy performs well on returns and maximum drawdown. Deep learning enables effective predictions. But deep structures face some challenges on human interpretability compared to conventional techniques, e.g., fuzzy inference systems. It motivates more research works to alleviate the black box nature of deep structures with performance maintained. This paper proposes a fuzzy-embedded recurrent neural network (FE-RNN) to improve interpretability of the underlying neural networks. It is a parallel deep structure comprising an RNN and a Pseudo Outer-Product based Fuzzy Neural Network (POPFNN) that share a common set of input and output linguistic concepts. The inference processes undertaken are associated by RNN using fuzzy rules in the embedded POPFNN. Fuzzy IF-THEN rules provide better interpretability of the inference process of the hybrid networks. It allows an effective realisation of a data driven implication using RNN in the modelling of fuzzy entailment within a fuzzy neural networks (FNN) structure. FE-RNN obtains more consistent results than other FNN in the experiment using the Mackey-Glass dataset. FE-RNN achieves about 99% correlation for forecasting prices of market indexes. Its interpretability is also discussed. FE-RNN then acts as a prediction tool in a financial trading system using forecast-assisted technical indicators optimised with Genetic Algorithms. It outperforms the benchmark trading strategies in the trading experiments. [ABSTRACT FROM AUTHOR]

Accurate privacy-preserving prediction using electronic health record (EHR) data distributed in multiple hospitals is essential to enable stakeholders related to healthcare services to obtain useful information without privacy leakage. In this paper, we propose harmonized embedding-based self-attentive encoder (HarmoSATE), which is a new method for privacy-preserving federated predictive analysis. We extract contextual embeddings of local institutions using Word2Vec, and then harmonize locally-trained embeddings using a neural network-based harmonization technique. The proposed method uses a deep representative encoder based on self-attention to learn complex and dynamic patterns inherent to harmonized embeddings of medical concepts. To evaluate our method, we implemented experiments using sequential medical codes collected from the Medical Information Mart for Intensive Care-III dataset in a distributed setting. It achieved a significant increase in average AUC, ranging from 3% to 8% depending on the experiments compared to baseline models, demonstrating superior prediction accuracy of a patient's diagnosis in the next admission. HarmoSATE can be a useful alternative to obtain accurate and practical results for various predictive tasks that use sensitive and distributed EHR data while preserving patients' privacy. [ABSTRACT FROM AUTHOR]

Progress in the field of Non-Intrusive Load Monitoring (NILM) has been attributed to the rise in the application of artificial intelligence. Nevertheless, the ability of energy disaggregation algorithms to disaggregate different appliance signatures from aggregated smart grid data poses some privacy issues. This paper introduces a new notion of disclosure risk termed energy disaggregation risk. The performance of Sequence-to-Sequence (Seq2Seq) NILM deep learning algorithm along with three activation extraction methods are studied using two publicly available datasets. To understand the extent of disclosure, we study three inference attacks on aggregated data. The results show that Variance Sensitive Thresholding (VST) event detection method outperformed the other two methods in revealing households' lifestyles based on the signature of the appliances. To reduce energy disaggregation risk, we investigate the performance of two privacy-preserving mechanisms based on microaggregation and Discrete Fourier Transform (DFT). Empirically, for the first scenario of inference attack on UK-DALE, VST produces disaggregation risks of 99%, 100%, 89% and 99% for fridge, dish washer, microwave, and kettle respectively. For washing machine, Activation Time Extraction (ATE) method produces a disaggregation risk of 87%. We obtain similar results for other inference attack scenarios and the risk reduces using the two privacy-protection mechanisms. [ABSTRACT FROM AUTHOR]

In this paper, we introduce ProNet, an novel deep learning approach designed for multi-horizon time series forecasting, adaptively blending autoregressive (AR) and non-autoregressive (NAR) strategies. Our method involves dividing the forecasting horizon into segments, predicting the most crucial steps in each segment non-autoregressively, and the remaining steps autoregressively. The segmentation process relies on latent variables, which effectively capture the significance of individual time steps through variational inference. In comparison to AR models, ProNet showcases remarkable advantages, requiring fewer AR iterations, resulting in faster prediction speed, and mitigating error accumulation. On the other hand, when compared to NAR models, ProNet takes into account the interdependency of predictions in the output space, leading to improved forecasting accuracy. Our comprehensive evaluation, encompassing four large datasets, and an ablation study, demonstrate the effectiveness of ProNet, highlighting its superior performance in terms of accuracy and prediction speed, outperforming state-of-the-art AR and NAR forecasting models. • Forecasting of electricity energy consumption and solar energy generation. • Informer-based model marrying forecasting horizon segmentation and variational inference techniques. • Comparing SARIMAX, DeepAR, DeepSSM, LogTrans, N-BEATS, and Informer models forecasting ability. • Performance evaluation based on four real-world datasets. [ABSTRACT FROM AUTHOR]

Network pruning has been a hot topic in recent years, and many popular pruning methods rely on network design expertise. However, the pruning process usually involves manual intervention and can be difficult for users who lack prior knowledge. Automatic pruning using evolutionary algorithms shows great promise, but it must address the challenge of performing time-consuming model evaluations and searching through a large solution space. Dataset distillation is a technique that compresses the original dataset to decrease the cost of fine-tuning models. In this paper, we explore the potential of using the distilled dataset to exhibit a similar role as the real dataset in network pruning, and proposed the evolutionary pruning framework using distilled dataset. Specifically, the network pruning pipeline is carried out on the distilled dataset to significantly reduce the model evaluation cost, and the number of filters in the convolutional layer is directly coded to narrow the search space. In addition, a tailored evolutionary algorithm is proposed that takes the form of constrained optimization to search the most suitable pruned network. The experiments conducted on VGG16, VGG19, ResNet56, and ResNet110 demonstrate that the proposed method reduces at least 41.56% of the flops and achieves competitive results with little compromising accuracy. • Distillation dataset is first applied to network pruning. • A constrained evolutionary algorithm with double-balanced multi-branch is proposed. • Deep convolutional neural network pruning is performed using the constrained evolutionary algorithm. [ABSTRACT FROM AUTHOR]

• Optimized deep learning, statistical and hybrid models are proposed for deterministic (point) and probabilistic (interval) forecasting of crude oil price (COP). • A DE-DL method employing differential evolution algorithm is proposed to optimize the architecture and other hyper-parameters of deep learning models for a problem. • Recurrence plot is drawn, recurrence quantification analysis is carried out and uncertainty analysis of prediction error is performed using Gaussian and t-Location Scale distributions. • Extensive non-parametric statistical analysis on the obtained results are carried out to rank the optimized models and draw decisive conclusions relating to deterministic and probabilistic forecasting of COP. • Eight research questions related to deterministic and probabilistic forecasting of COP are addressed. In this paper, individual and hybrid methods are proposed employing optimized statistical and deep learning (DL) models for deterministic (point) and probabilistic (interval) forecasting of crude oil price time series. The statistical models are optimized using the Forecast package of R. To enhance the performance of DL models, a novel pruning DE-DL method is proposed, which employs the differential evolution (DE) algorithm to optimize architecture and continuous and discrete-valued hyper-parameters. The proposed DE-DL method is so generic that it can be applied to optimize different DL models for any supervised learning problem. Five DL models (LSTM, BiLSTM, GRU, CNN, and ConvLSTM) are optimized for forecasting monthly crude oil prices and hybridized with an optimized ARIMA model for developing optimized additive and multiplicative hybrid forecasting models. The effectiveness of the proposed methods is evaluated through deterministic and probabilistic forecasting measures, comparing the results with six optimized statistical models, thirteen machine learning models, five optimized DL models, and ten optimized hybrid models. It is observed from the simulation results that the proposed optimized Additive-ARIMA-GRU hybrid model provides statistically superior forecasts, and the t Location Scale distribution is more suitable than the Gaussian distribution for computing reliable prediction intervals with different significance levels. [ABSTRACT FROM AUTHOR]

The evaluation of thermal comfort for railway passengers holds considerable importance, not only in reducing energy consumption but also in enhancing the passengers' experience. This paper presents a Transformer-based multi-task learning network (TransMTL) designed for railway passenger thermal comfort evaluation using EEG. We utilized manual features to extract temporal and frequency information, while a Transformer encoder distilled spatial information. The multi-task learning structure enhances model robustness by leveraging thermal comfort task correlations. We conducted experiments during winter and summer with high-speed railway passengers, establishing a comprehensive EEG dataset. The results demonstrated that our proposed EEG-TransMTL model outperformed classical machine learning and deep learning models in all four thermal comfort evaluation tasks, achieving accuracy rates of 65.00%, 66.70%, 80.38%, and 71.01%, respectively. We enhanced model interpretability by visualizing attention weights from the Transformer encoder, identifying key EEG channels. A simplified model utilizing only eight crucial channels also delivered notable performance. This research provides a practical and neuro-mechanism interpretable solution for thermal comfort evaluation. • The first EEG-based thermal comfort experiment in a train cabin was carried out. • A Transformer-based Multi-task Learning Network is proposed. • Our proposed method outperforms six baseline models in four tasks. • The crucial EEG channels are visualized and enhance the model's interpretability. [ABSTRACT FROM AUTHOR]

Multivariate time series classification is a crucial task with applications in broad areas such as finance, medicine, and engineering. Transformer is promising for time series classification, but as a generic approach, they have limited capability to effectively capture the distinctive characteristics inherent in time series data and adapt to diverse architectural requirements. This paper proposes a novel dynamic transformer-based architecture called Dyformer to address the above limitations of traditional transformers in multivariate time series classification. Dyformer incorporates hierarchical pooling to decompose time series into subsequences with different frequency components. Then, it employs Dyformer modules to achieve adaptive learning strategies for different frequency components based on a dynamic architecture. Furthermore, we introduce feature-map-wise attention mechanisms to capture multi-scale temporal dependencies and a joint loss function to facilitate model training. To evaluate the performance of Dyformer, we conducted extensive experiments using 30 benchmark datasets. The results unequivocally demonstrate that our model consistently outperforms a multitude of state-of-the-art methods and baseline approaches. Our model also copes well with limited training samples when pre-trained. • We present a transformer-based dynamic architecture to achieve adaptive learning strategies for different frequency components of the time series data. • We design a hierarchical pooling layer to decompose time series into subsequences representing different frequency components to facilitate time series classification. • We implement two types of gates for allowing the architecture to dynamically adapt the layer activation to realize a flexible structure. [ABSTRACT FROM AUTHOR]

Differential privacy (DP), as a promising privacy-preserving model, has attracted great interest from researchers in recent years. At present, research on the combination of deep learning and DP is active. In contrast, another widely used artificial intelligence technique, swarm intelligence (SI), has received little attention in the context of DP even though it also triggers privacy concerns. For this reason, this paper attempts to combine DP and SI for the first time and proposes a general differentially private swarm intelligence algorithm framework (DPSIAF). By utilizing the exponential mechanism, this framework can easily develop existing SI algorithms into the private versions. As examples, we apply it to four popular SI algorithms, and the related analyses demonstrate the effectiveness of our DPSIAF. More importantly, our private algorithms also exhibit some interesting experimental phenomena. In some cases, their performance is not strictly affected by the privacy budget, and one of them even outperforms its non-private version. These findings are different from the conventional cognition, which indicates DP owns the potential to serve as an optimization tool for some SI algorithms. Our study may provide a new perspective on DP and promote the synergy between metaheuristic optimization community and privacy computing community. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have tremendously succeeded in several computer vision-related fields. Nevertheless, previous research demonstrates that DNNs are vulnerable to adversarial sample attacks. Attackers add carefully designed perturbation noise to clean samples to form adversarial samples, which may lead to errors in the DNNs' predictions. Consequently, the safety of deep learning has attracted much attention, and researchers have commenced exploring adversarial samples from different perspectives. In this paper, a method based on class gradient networks (CGN) is proposed, which can generate high-quality adversarial samples by designing multiple objective functions. Specifically, the adversarial sample's high-level features are guided to change by introducing a high-level class gradient matrix, and the classification loss and perturbation loss are combined to jointly train a generator to fit the distribution of adversarial noises. We conducted experiments on two standard datasets, Fashion-MNIST and CIFAR-10. The results demonstrate the superiority of our method in the transferability of adversarial samples on targeted attacks and indicate the approach outperforms the baseline method. [ABSTRACT FROM AUTHOR]

Although deep learning models can solve complex prediction problems, they have been criticized for being 'black boxes'. This implies that their decisions are difficult, if not impossible, to explain by simply inspecting their internal knowledge structures. Explainable Artificial Intelligence has attempted to open the black-box through model-specific and agnostic post-hoc methods that generate visualizations or derive associations between the problem features and the model predictions. This paper proposes a new method, termed REPROT, that explains the decisions of complex deep learning architectures based on local reducts of an image. A 'reduct' is a set of sufficiently descriptive features that can fully characterize the acquired knowledge. The created reducts are used to build a 'prototype image' that visually explains the inference obtained by a black-box model for an image. We focus on deep learning architectures whose complexity and internal particularities demand adapting existing model-specific explanation methods, making the explanation process more difficult. Experimental results show that the black-box model can detect an object using the prototype image generated from the reduct. Hence, the explanations will be given by "the minimum set of features sufficient for the neural model to detect an object". The confidence scores obtained by architectures such as Inception, Yolo, and Mask R-CNN are higher for prototype images built from the reduct than those built from the most important superpixels according to the LIME method. Moreover, the target object is not detected on several occasions through the LIME output, thus supporting the superiority of the proposed explanation method. [ABSTRACT FROM AUTHOR]

Granular computing involves a comprehensive process that encompasses theories, methodologies, and techniques to solve complex problems, rather than being just an algorithm. As the volume of generated data continues to grow rapidly, data-driven problems have become increasingly complex. Although deep learning models have outperformed traditional machine learning models in solving complex problems, there is still room for enhancing their performance. In this paper, we propose a granular computing-based deep learning model, aimed at enhancing classifier accuracy in complex natural language-based problems. The proposed approach involves a new granulation method, which comprises a novel algorithm built on combinatorial concepts and ten rule-based numerical granules. By utilizing this granulation method, each granule adds a new representation and concept to the existing data. The proposed model consists of multiple models that perform learning separately in a granular view. In the final step, the model pays attention to the granulated matrices generated by various models. The proposed model is evaluated using datasets related to cyberbullying and two hate speech datasets, resulting in significant improvements in accuracy compared to state-of-the-art models. • Proposing a granular computing-based deep learning model for text classification. • Using granular computing for data augmentation from a new representation in the context of deep learning-based text classification. • Utilizing different representations of the existing texts. • Proposing the first stacked-BILSTM-SVM model in granular computing. [ABSTRACT FROM AUTHOR]