Search

Selecting discriminative features to build effective learning models is a significant research work in machine learning. In practical applications, the data distribution characteristics are diverse, and the uncertainties pose challenges for building learning models with robustness and generalization capabilities. Since one-hot encoding is good at representing independent labels, the label matrix of regression-based feature selection (FS) methods is usually encoded with one-hot encoding. However, it's not well adapted to the different data distributions. This paper proposes a sparse orthogonal supervised FS model with global redundancy minimization, label scaling, and robustness (GRMLSRSOFS) to address the above problems. This model uses the label scaling technique proposed in this paper to better adapt to different data distributions. An iterative optimization method is given, and its convergence is demonstrated theoretically and experimentally. Further, experimental results on 12 public datasets show that 1) The GRMLSRSOFS can achieve higher classification accuracy with fewer features in most cases than several state-of-the-art FS methods. For example, the GRMLSRSOFS achieves 100% classification accuracy using only 20 features on the warpPIE10P dataset and obtains nearly 6% improvement over other methods on the Yale dataset. 2) The convergence speed of the GRMLSRSOFS will be faster after label scaling. [ABSTRACT FROM AUTHOR]

• Based on the Euclidean distance between the input layer and the fuzzification layer, two sets of indicators are incorporated to assess the effectiveness and redundancy of current fuzzy rules. Rules determined as redundant are deleted. Meanwhile, new fuzzy rules are added when the current fuzzy rules become insufficient. Furthermore, a significance index is set to judge the essentiality of each fuzzy rule and remove unimportant rules. So that the algorithm can delete unimportant rules while deleting redundant rules. • The antecedent parameters of fuzzy rules are optimized by the AdaBound algorithm. By limiting the learning rate under a boundary value based on the Adam algorithm, this method has a higher training speed than the traditional gradient algorithm and simultaneously avoids the non-convergence caused by the extreme learning rate. • The adaptive gradient method is used to learn the consequences of fuzzy rules. By adjusting its learning rate with RMSE, this method can speed up the training speed while ensuring generalization capacity. • The proposed SIT2FWNN model is used to predict short-term traffic flow, Mackey-Glass chaotic time series, and the opening index of the Shanghai stock index and compared with similar studies. The simulation results demonstrate the superior network performance of the proposed SIT2FWNN in this paper. This paper presents a self-organizing interval type-2 fuzzy wavelet neural network (SIT2FWNN) model for predicting and identifying nonlinear systems. Based on traditional TSK interval type-2 fuzzy neural network (IT2FNN), the proposed SIT2FWNN utilizes the wavelet basis function with the locating abilities of time-domain and frequency-domain as the consequent of fuzzy rules, combining the capacity of IT2FNN to handle the uncertainty and the great learning potentiality of wavelet neural network (WNN). For the structural adjustment of SIT2FWNN, fuzzy rules are added and deleted according to the Euclidean distance between the input layer and the fuzzification layer. The self-organizing algorithm can delete redundant and unimportant fuzzy rules, thus optimizing the structure of SIT2FWNN and simplifying the calculation. In parameter learning, the AdaBound algorithm and self-adaptive gradient learning algorithm are used to find optimal values of unknown parameters of the SIT2FWNN model. Finally, the designed SIT2FWNN model is applied in the predictions of short-term traffic flow, Mackey-Glass time series, and the opening index of the Shanghai stock index. The evaluation comparison between the proposed model and similar studies proves that SIT2FWNN has higher prediction accuracy and speed. [ABSTRACT FROM AUTHOR]

Many black-box adversarial attack algorithms perform attacks on machine learning models based on the transferability of adversarial examples, and the input transformation-based attack is one of the most effective methods. However, existing input transformation methods ignore that each pixel contributes differently to the output of the model, and the focus regions of different models on the same images are similar. Therefore, this paper proposes a targeted data augmentation-based adversarial attack algorithm named MixCam, which augments the input data based on the contribution of each pixel to the prediction result. This is done to enhance the transferability of the crafted adversarial example from the perspective of shifting the regions to which the model pays most of its attention. In addition, this paper proposes further boosting the transferability of the crafted adversarial examples by fusing the class activation maps of multiple models for the input image. Furthermore, the MixCam can integrate other input transformation methods to further boost the transferability of crafted adversarial examples. Extensive experiments on ImageNet demonstrate that MixCam outperforms other state-of-the-art methods in black-box attacks against considered adversarially trained models, with an average increase of 11.7% and 10.7% in attack success rates for single and ensemble attack settings, respectively. • A targeted data augmentation-based black-box adversarial attack method. • Performing adversarial attacks from the perspective of moving the focus region of DNN. • Boosting the transferability of the crafted adversarial examples. • Extensive experiments have demonstrated the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Few-Shot Class-Incremental Learning (FSCIL)presents an extension of the Class Incremental Learning (CIL)problem where a model is faced with the problem of data scarcity while addressing the Catastrophic Forgetting (CF)problem. This problem remains an open problem because all recent works are built upon the Convolutional Neural Networks (CNNs)performing sub-optimally compared to the transformer approaches. Our paper presents Robust Transformer Approach (ROBUSTA)built upon the Compact Convolutional Transformer (CCT). The issue of overfitting due to few samples is overcome with the notion of the stochastic classifier, where the classifier's weights are sampled from a distribution with mean and variance vectors, thus increasing the likelihood of correct classifications, and the batch-norm layer to stabilize the training process. The issue of CFis dealt with the idea of delta parameters, small task-specific trainable parameters while keeping the backbone networks frozen. A non-parametric approach is developed to infer the delta parameters for the model's predictions. The prototype rectification approach is applied to avoid biased prototype calculations due to the issue of data scarcity. The advantage of ROBUSTAis demonstrated through a series of experiments in the benchmark problems where it is capable of outperforming prior arts with big margins without any data augmentation protocols. [ABSTRACT FROM AUTHOR]

The rapid integration of intelligent processes and methods into information systems in the Artificial Intelligence (AI) era has led to a substantial shift towards autonomous software decision-making. This evolution necessitates robust human oversight, especially in critical domains like Healthcare, Education, and Energy. Human trust in AI plays a vital role in influencing decision-making processes of users interacting with AI. This paper presents VIRTSI (V ariability and I mpact of R eciprocal T rust S tates towards I ntelligent systems), a novel rigorous computational model for human-AI Interaction. VIRTSI simulates human trust states, spanning from overtrust to distrust, through user modelling. It comprises: 1. A trust dynamics representational model based on Deterministic Finite State Automata (DFAs), illustrating transitions among cognitive trust states in response to AI-generated replies. 2. A trust evaluation model based on Confusion Matrices, originating from machine learning and Accuracy Metrics, providing a quantitative framework for analysing human trust dynamics. As a result, this is the first time that trust dynamics have been thoroughly traced in a representational model and a method has been developed to assess the impact of possibly harmful states like overtrust and distrust. An empirical study on the recently launched Large Language Model of generative AI, ChatGPT (version 3.5), provides a radical underexplored AI-generated platform for evaluating the human-AI interaction through VIRTSI. The study involved 1200 interactions of real users as well as AI experts together with experts in two very different domains of evaluation, namely software engineering and poetry. This study traces trust dynamics and the emerging human-AI interaction, in concrete examples of real user synergies with generative AI. The research reveals the vital role of maintaining normal trust states for optimal human-AI interaction and that both AI and human users need further steps towards this goal. The real-world implications of this research can guide the creation and evaluation of user interfaces with AI and the incorporation of functionalities in the development of generative AI chatbots in terms of trust by providing a new rigorous DFA representational method of trust dynamics and a corresponding new perspective of confusion matrix evaluation method of the dynamics' impact in the efficiency of human-AI dialogues. [ABSTRACT FROM AUTHOR]

Clustering, as an essential technique in unsupervised learning, plays a pivotal role in the fields of data mining and machine learning. However, the classic K -means clustering algorithm has intrinsic drawbacks such as sensitivity to initial cluster centers, susceptibility to a local optimal solution, and challenges in handling data uncertainty. To address these problems, this paper proposes an artificial hummingbird algorithm (AHA)-based three-way K -means clustering algorithm, called AHA-3WKM. First, AHA is introduced to address the problems of sensitivity to initial cluster centers and local optima. Second, a fitness function of AHA is specifically constructed to find the best initial clustering centers so that the hummingbirds can search for high-quality food sources, i.e., the global optimum cluster centers. Third, a three-way clustering approach is utilized to capture information about data uncertainty. In this way, the results of clustering are divided into three distinct regions based on the relationship between objects and clusters. The experimental results demonstrate that AHA-3WKM has good performance, and enhances the stability and the accuracy of clustering results. • AHA is introduced to address the problems of the sensitivity to initial cluster centers and the proneness to local optima. Hummingbirds are treated as data points, which dynamically update their strategies and effectively find the optimal cluster centers during multiple iterations. • A fitness function is designed based on the clustering principle of "birds of a feather flock together", with the aim of simplifying calculations, which enhances the specificity and practicality of K-means algorithm. • An AHA-based three-way K-means clustering algorithm (i.e., AHA-3WKM) is proposed. The clustering process is initialized with cluster centers optimized by AHA, and the results are represented in three regions, which can capture the uncertainty within the datasets. [ABSTRACT FROM AUTHOR]

Large-capacity machine learning models are vulnerable to membership inference attacks that disclose the privacy of the training dataset. The privacy concerns posed by membership inference attacks have inspired many defense strategies. Unfortunately, they have various limitations: 1) failing to achieve a high-quality tradeoff between membership privacy and model utility. 2) Providing little flexibility to users. This paper proposes Repair-Membership Learning (RM Learning), a simple but effective defense mechanism. RM Learning mitigates membership inference attacks by giving each sample of the training dataset an appropriate soft label to reduce the loss gap between the training and testing datasets of the model. Meanwhile, RM Learning controls the distinguishability between the training and testing datasets by limiting the model's loss gap over them with an adjustable parameter λ to give users flexibility. We evaluated the RM Learning algorithm on four datasets with seven models and compared it with three state-of-the-art methods. For instance, we reduce the effectiveness of black-box and white-box membership inference attacks to about 50.0%, with a loss of less than 1.3% of testing accuracy, for the CIFAR10 dataset with the ResNet18 model. [ABSTRACT FROM AUTHOR]

The use of artificial intelligence for decision support and automation has shown tremendous potential in many areas. The ability to explain the decisions made by a machine learning algorithm is fundamental to facilitating the widespread use of this type of tool. There are many important real-world problems where the cost of the decisions depends on the characteristics of each example: these are called example-dependent cost (EDC) problems. For this type of classification problem, an appropriate formulation that takes into account the decision costs is fundamental both for the design of the classifier and for the explanation of its decisions. In this paper, we propose COCOA, an explanation method designed for EDC problems based on a Bayesian discriminant. The proposed method can provide counterfactual samples generated by considering decision costs. The COCOA method provides valid and plausible counterfactuals with a high success rate, which can be actionable, diverse, and sparse, achieving a remarkable improvement in terms of cost over five state-of-the-art methods on six real-world datasets. • COCOA provides actionable, plausible, sparse and diverse counterfactuals. • Decision costs are taken into account when generating counterfactuals. • The obtained counterfactuals improve cost savings. • The method is presented for MLPs, but other architectures can be used. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of approximating functions from a Korobov space on [ − 1 , 1 ] d by ReLU shallow neural networks and present a rate O (m − 2 5 (1 + 2 d) log ⁡ m) of uniform approximation by networks of m hidden neurons. This is achieved by combining a novel Fourier analysis approach and a probability argument. We apply our approximation theory to a learning algorithm for regression based on ReLU shallow neural networks and derive learning rates of order O (N − 4 (d + 2) 9 d + 8 log ⁡ N) for the excess generalization error with the sample size N when the regression function lies in the Korobov space. [ABSTRACT FROM AUTHOR]

Attribute reduction is a crucial issue of rough set theory and has been applied to various fields. It aims to remove useless or redundant features from data and extract precise rules. Traditional attribute reduction methods have obvious limitations, that is, it can not guarantee the generalization ability of reduced decision rules to classification but can only guarantee the classification performance to visible objects. To overcome this issue and enhance the model classification generalization ability without increasing rules' complexity based attribute reduction, we propose a novel attribute method by employing the Structural Risk Minimization principle, which is a classic method in machine learning to balance model complexity and performance. The improved attribute reduction method with rough set tries to get a trade-off between the number of features and empirical error, wherein the former is defined by rule confidence, and the latter is explored by the mutual information between the condition attribute subset and the decision attribute. In other words, model accuracy can be regarded as an empirical error term to measure the model's classification or prediction performance, and the model complexity as a penalty term to characterize the size of the reduction subset. To implement this approach, genetic algorithm is employed as a heuristic search technique, to obtain the optimal reduction subset. Several comparative experiments are performed on ten UCI datasets to evaluate the model's classification accuracy and the size of the reduction subset. The experimental results indicate the method proposed in this paper with better generalization ability compared to other traditional algorithms under the condition of equal reducted set length. • We propose a new attribute reduction method by considering the model accuracy and complexity simultaneously. • We use rule confidence and mutual information to define the model's accuracy and complexity, respectively. • We employ the genetic algorithm as a heuristic search technique to obtain the optimal reducted subset. [ABSTRACT FROM AUTHOR]

Feature selection has become a mandatory task in data mining, due to the overwhelming amount of features in Big Data problems. To handle this high-dimensional data and avoid the well-known curse of dimensionality, we need to pre-select an optimal subset of features to reduce redundant computations. Federated learning is a machine learning technique based on training an algorithm over many decentralized edge devices holding local rather than global data on a centralized server. Application of this technique is extending to fields such as self-driving cars, medicine and health, and Industry 4.0, where data privacy is compulsory. Feature selection through federated learning is a complicated task since suboptimal features calculated by feature selection methods may be different in heterogeneous datasets from different nodes. In this paper, we propose a lossless federated version of the classic minimum redundancy maximum relevance (mRMR) feature selection algorithm, called federated mRMR (fed-mRMR), which, without losing any effectiveness of the original mRMR method, is applicable to federated learning approaches and capable of dealing with data that are not independent and identically distributed (non-IID data). Implementation can be found at: https://github.com/jorgehermo9/fed-mrmr • Feature Selection (FS) is a key step in machine learning. • Classical FS does not consider privacy preservation. • Federated FS allows to perform feature selection in federated environments. • fed-mRMR is a federated feature selection tool publicly available. • fed-mRMR performs better than original's mRMR proposal in terms of computation. [ABSTRACT FROM AUTHOR]

This paper introduces a novel machine learning approach, called BBATDD (Boosting-Based Algorithm Trained with Drift Detector), that departs from traditional epoch-based methods by adopting a data stream generation strategy from the original dataset. The proposed method is versatile, applicable to a broad range of deep structures and other machine learning architectures. The key to its effectiveness is the conversion of original data into a streaming sequence, which is then fed into the neural network structure. To monitor the learning process and prevent overfitting, a drift detector, commonly used in stream data mining, is employed. Additionally, two methods, ELB and ENT, are introduced to prioritize problematic data elements during the learning process, enhancing overall model performance. Extensive simulations on benchmark datasets (MNIST and CIFAR) validate that BBATDD achieves higher accuracy with significantly fewer batches, without sacrificing generalization capability. For instance, when applied to the MNIST dataset and processing 10 000 batches while comparing loss function values, our approach demonstrates a remarkable 45.6% improvement. [ABSTRACT FROM AUTHOR]

The k Nearest Neighbors (k NN) search in high-dimensional space is a fundamental problem with various applications. In this paper, we try to solve this problem by using deep neural networks (DNNs). We apply DNNs to represent complex correlations between high-dimensional objects, enabling us to project similar objects into the same class, thus reducing the search cost. Based on DNNs, we propose two novel techniques to improve query efficiency while keeping accuracy. First, traditional DNNs typically demand extensive training data for achieving high accuracy. To decrease the training size, we design a multi-module DNN framework comprising several small modules. Each module learns to capture part of knowledge for the given query. The collective output of these sub-modules is then seamlessly integrated to form the final result. Second, with machine learning models, the size of candidates are unbounded. Thus, we design a linear-time data layout refinement algorithm, aiming to limit the number of candidates to a small constant. Empirically we find that our approach significantly outperforms the state-of-the-art methods in terms of both time efficiency and space efficiency while still attaining comparable or better accuracy. [ABSTRACT FROM AUTHOR]

In a modern distributed learning framework, the speeds of intra-worker calculation and inter-worker communication may differ by 1,000 times. It is advisable to perform expensive calculations on worker machines and communicate as few rounds as possible. In this paper, we propose three novel distributed sparse learning algorithms (EDSL-ET, EDSL-IBCD, and EDSL-IBCD-ET) in high dimensions, which are efficient in both computation and communication. Algorithm EDSL-ET utilizes the Top k sparse technique to reduce communication costs and adopts error feedback to guarantee convergence. Algorithm EDSL-IBCD greatly reduces communication costs by introducing an independent block coordinate descent method. Algorithm EDSL-IBCD-ET takes advantage of EDSL-ET and EDSL-IBCD algorithms for extra-high dimensional feature learning, which is the best one whether in communication, computation, and stability. We give theoretical guarantees of the three algorithms under mild conditions, which match the performance of the centralized algorithm. Extensive experiments on simulated and real data validate our theoretical analysis and demonstrate that the proposed algorithms perform well with just a few rounds of communications. [ABSTRACT FROM AUTHOR]

Hybrid learning is an excellent method that combines the global information of data with the local information of data. Different from the known hybrid learning algorithms, in this paper we propose a new hybrid learning strategy for Fisher Linear Discriminant (FLD) and introduce novel hybrid learning based on FLD (HL-FLD). The main idea of HL-FLD is to obtain the global structural information by FLD firstly, and then use the obtained global information to divide the given data locally in a more detailed way. To study systematically the proposed HL-FLD, we not only establish the generalization bound of HL-FLD and prove that the proposed HL-FLD algorithm is consistent, but also present some discussions on HL-FLD. Since splitting the blocks of a given data is a hyperparameter, we also improve HL-FLD and introduce another new self-adaptive hybrid learning based on FLD (SHL-FLD). The experimental researches for benchmark repository confirm that the proposed two algorithms have better performance in terms of misclassification rates and total time. [ABSTRACT FROM AUTHOR]

Rough set theory is a well-known mathematical framework that can deal with inconsistent data by providing lower and upper approximations of concepts. A prominent property of these approximations is their granular representation: that is, they can be written as unions of simple sets, called granules. The latter can be identified with "if..., then..." rules, which form the backbone of rough set rule induction. Alternatively, this property of granular representability can be defined as being free from inconsistencies, i.e. there are no elements within the set that are indiscernible to elements outside the set. It has been shown previously that this property can be maintained for various fuzzy rough set models, including those based on ordered weighted average (OWA) operators. In this paper, we will focus on some instances of the general class of fuzzy quantifier-based fuzzy rough sets (FQFRS). In these models, the lower and upper approximations are evaluated using binary and unary fuzzy quantifiers, respectively. One of the main targets of this study is to examine the granular representation of different models of FQFRS. The main findings reveal that Choquet-based fuzzy rough sets can be represented granularly under the same conditions as OWA-based fuzzy rough sets, whereas Sugeno-based FRS can always be represented granularly. Additionally, we provide counterexamples for the granular representability of other FQFRS models, and show that despite this, these approaches still demonstrate effectiveness in mitigating inconsistencies in real-life datasets. This observation highlights the potential of these models for resolving data inconsistencies and managing noise. [ABSTRACT FROM AUTHOR]

Cloud-edge architecture is an emerging technology that aims to meet the growing demands of intelligent applications. To address the issues of machine learning privacy leakages and benefiting from imbalanced data distribution, federated learning has been widely applied. Nevertheless, they present inherent vulnerabilities that make them vulnerable to poisoning attacks. Existing defense techniques are largely attack-rigid: they are designed to recognize client properties or model updates directly, aimed at specific attack scenarios or rules, but may not work well for critical feature patterns or flexible attack methods, mainly due to the potential influence of redundant features and model performance on defense. Yet few flexible defense methods have been developed. In this paper, we propose FlexibleFL, a flexible defense method against poisoning attacks in cloud-edge federated learning system (CEFL). The key idea of FlexibleFL is to evaluate the quality of uploaded model parameters and further determine the contribution of participants through an optimal threshold selection strategy. Based on these differences, FlexibleFL can thus implement penalties to potential attackers in a way that involves assigning the updated federated model. Extensive results demonstrate that our method has significant advantages in countering poisoning attacks in IID and Non-IID scenarios, and can effectively protect CEFL systems. • Design the CEFL framework to provide protection for distributed training while exploring its inherent vulnerabilities. • Propose FlexibleFL defense method with the aim of evaluating the contributions of the participants. • Introduce the optimal threshold selection strategy to balance the model performance and defense effectiveness. • FlexibleFL exhibits advantages and flexibility against poisoning attacks in both IID and Non-IID scenarios. [ABSTRACT FROM AUTHOR]

Unsupervised feature selection (UFS) is a fundamental task in machine learning and data analysis, aimed at identifying a subset of non-redundant and relevant features from a high-dimensional dataset. Embedded methods seamlessly integrate feature selection into model training, resulting in more efficient and interpretable models. Current embedded UFS methods primarily rely on self-representation or pseudo-supervised feature selection approaches to address redundancy and irrelevant feature issues, respectively. Nevertheless, there is currently a lack of research showcasing the fusion of these two approaches. This paper proposes the Orthogonal Encoder-Decoder factorization for unsupervised Feature Selection (OEDFS) model, combining the strengths of self-representation and pseudo-supervised approaches. This method draws inspiration from the self-representation properties of autoencoder architectures and leverages encoder and decoder factorizations to simulate a pseudo-supervised feature selection approach. To further enhance the part-based characteristics of factorization, orthogonality constraints and local structure preservation restrictions are incorporated into the objective function. The optimization process is based on the multiplicative update rule, ensuring efficient convergence. To assess the effectiveness of the proposed method, comprehensive experiments are conducted on 14 datasets and compare the results with eight state-of-the-art methods. The experimental results demonstrate the superior performance of the proposed approach in terms of UFS efficiency. [ABSTRACT FROM AUTHOR]

Fake news detection (FND) tools are essential to increase the reliability of information in social media. FND can be approached as a machine learning classification problem so that discriminative features can be automatically extracted. However, this requires a large news set, which in turn implies a considerable amount of human experts' effort for labeling. In this paper, we explore Positive and Unlabeled Learning (PUL) to reduce the labeling cost. In particular, we improve PUL with the network-based Label Propagation (PU-LP) algorithm. PU-LP achieved competitive results in FND exploiting relations between news and terms and using few labeled fake news. We propose integrating an attention mechanism in PU-LP that can define which terms in the network are more relevant for detecting fake news. We use GNEE, a state-of-the-art algorithm based on graph attention networks. Our proposal outperforms state-of-the-art methods, improving F 1 in 2% to 10%, especially when only 10% labeled fake news are available. It is competitive with the binary baseline, even when nearly half of the data is labeled. Discrimination ability is also visualized through t-SNE. We also present an analysis of the limitations of our approach according to the type of text found in each dataset. [ABSTRACT FROM AUTHOR]

Recent research has shown that the unaware automation of high-risk decision-making tasks can result in unfair decisions being made. The most common approaches to address this problem adopt definitions of fairness based on protected attributes. Precise annotation of protected attributes enables the application of bias mitigation techniques to commonly unlabeled kinds of data (e.g., images, text, etc.). This paper proposes a framework to automatically annotate protected attributes in data collections. The framework focuses on providing a single interface to annotate protected attributes of different types (e.g., gender, race, etc.) and from different kinds of data. Internally, the framework coordinates multiple sensors to produce the final annotation. Several sensors for textual data are proposed. An optimization search technique is designed to tune the framework to specific domains. Additionally, a small dataset of movie reviews —annotated with gender and sentiment— was created. The evaluation in datasets of texts from diverse domains shows the quality of the annotations and their effectiveness to be used as a proxy to estimate fairness in datasets and machine learning models. The source code is available online for the research community. • A framework to automatically annotate protected attributes in datasets. • Techniques to annotate gender in textual collections with fairness considerations. • Optimization search approach for tuning the framework to custom domains. • Small dataset of movie reviews annotated with gender and sentiment. • Effective use of annotations as a proxy to estimate fairness in datasets. [ABSTRACT FROM AUTHOR]

Algorithm selection technology, aimed at choosing the most suitable algorithm for a given machine learning task, has achieved significant success in the domain of 2D vision. However, few studies have explored its application to the 3D point cloud domain. The rapid proliferation and development of 3D point cloud classification algorithms underscore the urgent need for exploration in selecting these algorithms. In this paper, we propose a novel meta-learning-based 3D classification approach, termed 3D meta-classification , to address this gap. The approach operates at both the base-level and meta-level phases. At the base level, candidate 3D classification algorithms constantly classify various 3D datasets, recording their classification performance as empirical knowledge. As for the meta-level phase, it specifically tailors the 3D meta-knowledge generator, 3D meta-feature extractor, and 3D meta-database constructor to establish the 3D meta-database, capturing the relationship between the 3D meta-features and empirical knowledge. Leveraging the 3D meta-database, the 3D classification meta-learner trains the meta-model and predicts suitable algorithms for new incoming 3D datasets. Extensive experiments are conducted to select the best-performing classification algorithm for specific 3D datasets from ModelNet40. The results demonstrate the effectiveness of the proposed 3D meta-classification model; the accuracies of one-from-two and one-from-three algorithm selection tasks reach over 98% and 80%, respectively. [ABSTRACT FROM AUTHOR]

Feature selection has been widely used in machine learning for a long time. In this paper, we propose a supervised local sparse discriminative feature selection method named LSDFS to obtain sparse features by imposing ℓ 2 , 0 -norm constraint on transformation matrix. Differently from traditional approaches, our method does not require approximation or relaxation schemes, such as ℓ 2 , p -norm to solve long-standing challenge. Our method is based on the trace difference form of Linear Discriminant Analysis (LDA), which can efficiently obtain discriminative information in low-dimensional space. In order to explore the local structure of data which contains more discriminative information, we adopt a sparse connections graph between anchor points and data points instead of fully-connected graph with time-consuming, and add a decay parameter to avoid trivial solutions, making the model more precisely. Extensive experiments conducted on synthetic datasets and several real-world datasets have demonstrated the advantages of our method. [ABSTRACT FROM AUTHOR]

Progress in the field of Non-Intrusive Load Monitoring (NILM) has been attributed to the rise in the application of artificial intelligence. Nevertheless, the ability of energy disaggregation algorithms to disaggregate different appliance signatures from aggregated smart grid data poses some privacy issues. This paper introduces a new notion of disclosure risk termed energy disaggregation risk. The performance of Sequence-to-Sequence (Seq2Seq) NILM deep learning algorithm along with three activation extraction methods are studied using two publicly available datasets. To understand the extent of disclosure, we study three inference attacks on aggregated data. The results show that Variance Sensitive Thresholding (VST) event detection method outperformed the other two methods in revealing households' lifestyles based on the signature of the appliances. To reduce energy disaggregation risk, we investigate the performance of two privacy-preserving mechanisms based on microaggregation and Discrete Fourier Transform (DFT). Empirically, for the first scenario of inference attack on UK-DALE, VST produces disaggregation risks of 99%, 100%, 89% and 99% for fridge, dish washer, microwave, and kettle respectively. For washing machine, Activation Time Extraction (ATE) method produces a disaggregation risk of 87%. We obtain similar results for other inference attack scenarios and the risk reduces using the two privacy-protection mechanisms. [ABSTRACT FROM AUTHOR]

Federated Learning is a distributed machine learning method that offers inherent advantages in efficient learning and privacy protection within edge computing scenarios. However, terminal nodes often encounter challenges such as insufficient datasets and a significant amount of unlabelled data, leading to reduced accuracy in multi-party collaborative training models. Prior approaches have typically relied on a single pseudo label from unlabelled data to guide model training, limiting the utilization of knowledge within these data. To address this, this paper proposes a federated semi-supervised learning method (FedTG) tailored for image classification. Specifically, we leverage multiple high probability pseudo labels from unlabelled data to participate in semi-supervised learning, rather than relying on a single pseudo label. This approach mitigates the potential harm caused by errors in a single pseudo label and enables the model to fully capture the knowledge within the unlabelled data. Additionally, recognizing the significance of model classifiers (final neural network layer) in image classification tasks, we propose the exclusion of model classifier updates during the training process using unlabelled data to maintain optimal classification performance. Experiments conducted on real datasets have demonstrated that the FedTG method effectively enhances the accuracy of traditional Federated Learning model. [ABSTRACT FROM AUTHOR]

Machine Unlearning is a recently proposed paradigm to make an machine learning (ML) model delete specific data. Specifically, the data owner has the right to ask the machine learning as a service (MLaaS) provider to remove the impact of specific data on the trained model, so as to protect the privacy of the forgotten data. However, in order to achieve the desired effect, the unlearning operation does come at a certain cost. Therefore, the dishonest MLaaS provider has an incentive to create fake forgetting feedback to deceive the data owner's unlearning verification without performing any unlearning operations. The primary objective of the paper is to understand the potential vulnerabilities within machine unlearning mechanisms and contribute to the development of more robust, trustworthy, and secure unlearning solutions. We propose the concept of Pseudo Unlearning for the first time, and designs an efficient scheme, s ample s wapping with h ash (SSH), for the membership inference attack verification mechanism. We conduct extensive experiments on different datasets, and the performance achieved in the evaluation metrics as well as the verification mechanisms of membership inference attack shows the feasibility of pseudo unlearning. [ABSTRACT FROM AUTHOR]

Natural language processing (NLP) backdoor attacks have become a hidden threat to modern NLP applications. Most of the existing defense methods defend against specific types of backdoor attacks, and they generally fail to defend against invisible backdoors with syntactically correct triggers. This paper proposes NLPSweep, a comprehensive defense scheme that can defend against five common types of backdoor attacks, namely, character, word, sentence, homograph, and learnable textual attacks. Specifically, we propose a framework that can discover an effective defense solution without prior knowledge of the attacks. The defense solution is optimized from the framework and can defend against various attacks while ensuring high accuracy. Finally, we verify the effectiveness of NLPSweep on two pretrained models (BERT and XLNET) on three classic datasets (SST-2, IMDB, and OLID) and compare it with five state-of-the-art defense methods, namely, ONION, Pred, RAP, Fine-pruning, and STRIP. The experimental results demonstrate that NLPSweep has an average model accuracy (ACC) greater than 0.922 and that the average attack success rate (ASR) is only 0.202, outperforming the compared methods. Furthermore, NLPSweep is tested on the real-world Yelp dataset and it can effectively defend against backdoor attacks with the ASR less than 0.07 and the ACC greater than 0.973.1 • Existing defense methods only defend against specific backdoor attacks. • Our defense framework can discover an effective defense solution without prior knowledge of the attacks. • Our defense solution can defend against various attacks while ensuring high accuracy. • Tests on three classic datasets show that our defense scheme outperforms recent work. • Our scheme demonstrates operational success on the real-world Yelp dataset, effectively defending against backdoor attacks. [ABSTRACT FROM AUTHOR]

Federated learning is a commonly distributed framework for large-scale learning, where a model is learned over massively distributed remote devices without sharing information on devices. It has at least three key challenges: heterogeneity in federated networks, privacy and communication costs. In this paper, we propose three federated learning algorithms to handle these issues gradually. First, we introduce a FedSfDane algorithm (DANE with S hrinkage f actor for Fed erated learning), which improves the inexact approximation of the full gradient, captures statistical heterogeneity and restrains systems heterogeneity across the devices. For avoiding possible privacy leakage in federated learning, a P rivacy-preserving FedSfDane algorithm (PFedSfDane) is proposed, which is resistant to adversary attacks. Further, we give a novel C ommunication-efficient PFedSfDane (CPFedSfDane) algorithm for large-scale federated networks, which effectively handles the above three challenges. We give convergence guarantees for the three algorithms to convex and non-convex learning problems. Numerical experiments illustrate our algorithms outperform FedDANE, FedAvg and FedProx algorithms, especially for highly heterogeneous federated networks. CPFedSfDane improves the prediction accuracy of the state-of-the-art FedDANE algorithm by about 15.0% on sent140 dataset, and has high privacy protection and communication efficiency. [ABSTRACT FROM AUTHOR]

Interval type-2 fuzzy neural networks (IT2FNNs) have gained sustainable attention and wide applications because of their power of adaptive fuzzy modeling. Although the existing models accurately simulate the relationships between inputs and outputs by various adaptive learning algorithms, the parameters and fuzzy rules characterizing the model behaviors no longer have explicit interpretations. To achieve good interpretability and generalization performance, this paper proposes a novel data-knowledge-driven IT2FNN (DKIT2FNN). Unlike accuracy-oriented model fitting, DKIT2FNN pioneers the use of famous association rule mining techniques to automatically explore the interpretable rule knowledge, which intuitively presents conditional relation patterns among features. Using the obtained knowledge, the DKIT2FNN model is accomplished in a single-hidden layer network framework, in which the antecedent structure of each fuzzy rule is determined by the relation patterns provided by rule knowledge. In addition, an adversarial incremental learning algorithm is proposed for the self-organizing modeling of DKIT2FNN. Besides automatically assembling a compact rule base, it integrates the label adversarial training to reinforce the model's capability against label noise. We validate the DKIT2FNN on classification benchmark datasets with label noise and compare it with state-of-the-art non-fuzzy and neuro-fuzzy methods. The results demonstrate that the proposed DKIT2FNN achieves robust classification performance and good interpretability. • Propose a data-knowledge-driven interval type-2 fuzzy neural network (DKIT2FNN). • Realize an automated knowledge mining to ensure reliable fuzzy rule interpretation. • Develop an incremental learning algorithm integrating label adversarial training. • DKIT2FNN achieves a good and robust generalization performance facing label noise. • DKIT2FNN can self-organize a compact model structure with good interpretability. [ABSTRACT FROM AUTHOR]

The random generation of fuzzy measures under complex linear constraints holds significance in various fields, including optimization solutions, machine learning, decision making, and property investigation. However, most existing random generation methods primarily focus on addressing the monotonicity and normalization conditions inherent in the construction of fuzzy measures, rather than the linear constraints that are crucial for representing special families of fuzzy measures and additional preference information. In this paper, we present two categories of methods to address the generation of linearly constrained fuzzy measures using linear programming models. These methods enable a comprehensive exploration and coverage of the entire feasible convex domain. The first category involves randomly selecting a subset and assigning measure values within the allowable range under given linear constraints. The second category utilizes convex combinations of constrained extreme fuzzy measures and vertex fuzzy measures. Then we employ some indices of fuzzy measures, objective functions, and distances to domain boundaries to evaluate the coverage performance of these methods across the entire feasible domain. We further provide enhancement techniques to improve the coverage ratios. Finally, we discuss and demonstrate potential applications of these generation methods in practical scenarios. [ABSTRACT FROM AUTHOR]

• Optimized deep learning, statistical and hybrid models are proposed for deterministic (point) and probabilistic (interval) forecasting of crude oil price (COP). • A DE-DL method employing differential evolution algorithm is proposed to optimize the architecture and other hyper-parameters of deep learning models for a problem. • Recurrence plot is drawn, recurrence quantification analysis is carried out and uncertainty analysis of prediction error is performed using Gaussian and t-Location Scale distributions. • Extensive non-parametric statistical analysis on the obtained results are carried out to rank the optimized models and draw decisive conclusions relating to deterministic and probabilistic forecasting of COP. • Eight research questions related to deterministic and probabilistic forecasting of COP are addressed. In this paper, individual and hybrid methods are proposed employing optimized statistical and deep learning (DL) models for deterministic (point) and probabilistic (interval) forecasting of crude oil price time series. The statistical models are optimized using the Forecast package of R. To enhance the performance of DL models, a novel pruning DE-DL method is proposed, which employs the differential evolution (DE) algorithm to optimize architecture and continuous and discrete-valued hyper-parameters. The proposed DE-DL method is so generic that it can be applied to optimize different DL models for any supervised learning problem. Five DL models (LSTM, BiLSTM, GRU, CNN, and ConvLSTM) are optimized for forecasting monthly crude oil prices and hybridized with an optimized ARIMA model for developing optimized additive and multiplicative hybrid forecasting models. The effectiveness of the proposed methods is evaluated through deterministic and probabilistic forecasting measures, comparing the results with six optimized statistical models, thirteen machine learning models, five optimized DL models, and ten optimized hybrid models. It is observed from the simulation results that the proposed optimized Additive-ARIMA-GRU hybrid model provides statistically superior forecasts, and the t Location Scale distribution is more suitable than the Gaussian distribution for computing reliable prediction intervals with different significance levels. [ABSTRACT FROM AUTHOR]

Conversational emotion recognition represents an important machine-learning problem with a wide variety of deployment possibilities. The key challenge in this area is how to properly capture the key conversational aspects that facilitate reliable emotion recognition, including utterance semantics, temporal order, informative contextual cues, speaker interactions as well as other relevant factors. In this paper, we present a novel Graph Neural Network approach for conversational emotion recognition at the utterance level. Our method addresses the outlined challenges and represents conversations in the form of graph structures that naturally encode temporal order, speaker dependencies, and even long-distance context. To efficiently capture the semantic content of the conversations, we leverage the zero-shot feature-extraction capabilities of pre-trained large-scale language models and then integrate two key contributions into the graph neural network to ensure competitive recognition results. The first is a novel context filter that establishes meaningful utterance dependencies for the graph construction procedure and removes low-relevance and uninformative utterances from being used as a source of contextual information for the recognition task. The second contribution is a feature-correction procedure that adjusts the information content in the generated feature representations through a gating mechanism to improve their discriminative power and reduce emotion-prediction errors. We conduct extensive experiments on four commonly used conversational datasets, i.e., IEMOCAP, MELD, Dailydialog, and EmoryNLP, to demonstrate the capabilities of the developed graph neural network with context filtering and error-correction capabilities. The results of the experiments point to highly promising performance, especially when compared to state-of-the-art competitors from the literature. [ABSTRACT FROM AUTHOR]

Accurate estimation of energy savings is crucial for the effective implementation of energy conservation measures (ECMs). Simultaneously, the integration of Artificial Intelligence (AI) has revolutionized software engineering by imbuing software with intelligent capabilities and autonomy. In this paper, we propose an ensemble model for precisely estimating baseline energy consumption within the realm of AI-empowered autonomous software. The ensemble model combines predictions from three tree-based Machine Learning models, namely Random Forest, XGBoost, and LightGBM. Notably, our model emphasizes the provision of explainability, granting transparency and insights into the key factors influencing baseline energy consumption. To validate its effectiveness, we conduct experimental evaluations on a diverse cluster of real-world buildings in Latvia. The results demonstrate the superiority of our proposed ensemble model over individual models and even a deep learning network tailored for energy consumption estimation. These findings underscore the efficacy of AI-empowered models in the energy sector, offering a robust and interpretable solution for estimating energy savings. [ABSTRACT FROM AUTHOR]

Optimal granularity selection is a key issue in rough set, by which decision rules can be generated to assign corresponding decision labels for new samples. The current evaluation criteria of optimal granularity mainly focus on the experimental effect of prediction algorithms, thus ignoring the theoretical relationship between granularity and the generalization ability of algorithms. In this paper, we study the issue overlooked above and propose a novel framework for optimal granularity selection with theoretical guarantee. In our framework, a rough set-based prediction algorithm that incorporates a global confidence as scoring function is introduced, which is characterized by the granularity-based loss function. On the bias, the relationship between granularity and the generalization ability of algorithm is studied by introducing the stability theory in machine learning. The generalization error bound of algorithm is derived as a theoretical guarantee for optimal granularity selection. Finally, a novel optimal granularity selection strategy with theoretical guarantee is proposed, which is combined with existing attribute reduction methods to design the attribute re-reduction algorithms for generate optimal granularity by reducing the relatively unimportant attributes. Numerical experiments verify the rationality of optimal granularity selection framework and prove the effectiveness of the optimal granularity generated by the attribute re-reduction algorithms. [ABSTRACT FROM AUTHOR]

Federated learning is a promising approach in the Internet of Things (IoT) that enables collaborative and distributed machine learning among massive IoT devices without sharing private data, thereby constructing smart IoT applications. However, traditional federated learning approaches are unable to monitor the local training process of devices. Some malicious devices can exploit the vulnerability to conduct Byzantine attacks, which may potentially lead to the failure or compromise of shared global model. In this paper, we propose a new Byzantine-robust federated learning framework called rFedFW, which aims to achieve secure and trustable federated learning in the IoT. Specifically, we propose a dual filtering mechanism to identify and discard malicious gradients. Furthermore, we design an adaptive weight adjustment scheme that dynamically reduces the aggregated weight of potentially malicious gradients, ultimately achieving robust model aggregation. Additionally, we propose a dynamic clipping method to reduce the magnitude of various gradients, and we incorporate an additive model aggregation scheme with momentum to smooth the effects of local gradients and achieve efficient model aggregation. Extensive experimental results on various datasets demonstrate the effectiveness and robustness of rFedFW. [ABSTRACT FROM AUTHOR]

Practitioners use feature importance to rank and eliminate weak predictors during model development in an effort to simplify models and improve generality. Unfortunately, they also routinely conflate such feature importance measures with feature impact, the isolated effect of an explanatory variable on the response variable. This can lead to real-world consequences when importance is inappropriately interpreted as impact in applications like medicine and business. The dominant approach for computing feature importance is through interrogation of a fitted model, which works well for feature selection, but gives distorted measures of feature impact. For example, the same method applied to the same data set can yield different feature importances, depending on the model, leading us to conclude that impact should be computed directly from the data. While there are nonparametric feature selection algorithms, they typically provide feature rankings, rather than direct measures of impact or importance. They also often focus on single-variable associations with the response. In this paper, we provide mathematical definitions of feature impact and importance, derived from partial dependence curves, that operate directly on the data. We develop two methods, StratImpact and StratImp, that estimate feature impact and importance from partial dependence measures using stratification of the explanatory variables. We show that features ranked by these definitions are competitive with, and often better than, existing feature selection techniques. We validate our methods through a comparison with contemporary methods using three real data sets and a testbed of simulated data. [ABSTRACT FROM AUTHOR]

Granular computing involves a comprehensive process that encompasses theories, methodologies, and techniques to solve complex problems, rather than being just an algorithm. As the volume of generated data continues to grow rapidly, data-driven problems have become increasingly complex. Although deep learning models have outperformed traditional machine learning models in solving complex problems, there is still room for enhancing their performance. In this paper, we propose a granular computing-based deep learning model, aimed at enhancing classifier accuracy in complex natural language-based problems. The proposed approach involves a new granulation method, which comprises a novel algorithm built on combinatorial concepts and ten rule-based numerical granules. By utilizing this granulation method, each granule adds a new representation and concept to the existing data. The proposed model consists of multiple models that perform learning separately in a granular view. In the final step, the model pays attention to the granulated matrices generated by various models. The proposed model is evaluated using datasets related to cyberbullying and two hate speech datasets, resulting in significant improvements in accuracy compared to state-of-the-art models. • Proposing a granular computing-based deep learning model for text classification. • Using granular computing for data augmentation from a new representation in the context of deep learning-based text classification. • Utilizing different representations of the existing texts. • Proposing the first stacked-BILSTM-SVM model in granular computing. [ABSTRACT FROM AUTHOR]

Current machine learning algorithms encounter challenges such as missing labels and high dimensionality. Feature selection serves as an effective dimensionality reduction technique, enhancing the efficiency and accuracy of subsequent machine learning tasks by eliminating irrelevant and redundant features. Given the difficulty in obtaining fully labeled data, partially labeled data has become a crucial target for machine learning models to address. The related family is an efficient, rough set-based feature selection approach; however, it cannot be applied to semi-supervised learning tasks. Consequently, this paper introduces a semi-supervised feature selection method based on a fuzzy related family for partially labeled data. At first, the fuzzy label values of unlabeled samples are calculated based on fuzzy similarity relationships by establishing a novel fuzzy covering system. Subsequently, a fuzzy related family is constructed by a consistent fuzzy set. Then a semi-supervised feature selection algorithm, referred to as the Semi-supervised Fuzzy Related Family (SFRF), is developed using the established feature significance measurement. Compared to existing semi-supervised feature selection algorithms, SFRF considerably enhances feature selection efficiency while preserving classification accuracy. Specifically, the average reduction efficiency across twelve datasets increased by up to 109 times. [ABSTRACT FROM AUTHOR]