Search

Intrusion Detection System (IDS) can ensure the network security by identifying network intrusions according to the abnormal traffic data. However, the intrusion detection data has the problem of high dimensionality and changes with network and attack environments, which leads to the poor performance and poor portability of intrusion detection algorithms. Therefore, this paper proposes an intrusion detection algorithm based on joint symmetric uncertainty and hyperparameter optimized fusion neural network. Firstly, a feature selection method based on symmetric uncertainty and approximate Markov blanket is proposed, which fully considers the correlation and redundancy of features, and also the correlation between combined features and the class label, so as to reduce the data dimensionality. Secondly, the CNN-LSTM classifier fused with Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) is used to extract the spatial features and temporal features to improve the classification performance. Finally, the Particle Swarm Optimization (PSO) algorithm is improved and used to automatically optimize the hyperparameters of the classifier, so that the classifier can be applied to different intrusion detection datasets with better generalization ability and portability. Experiments have verified the effectiveness and superiority of the proposed algorithm on multiple evaluation indicators. • An effective algorithm for intrusion detection is proposed in this paper. • Feature selection is based on symmetric uncertainty and approximate Markov blanket. • A fusion neural network is constructed to extract the spatial and temporal features. • The PSO algorithm is improved to automatically optimize the hyperparameters. [ABSTRACT FROM AUTHOR]

• A feature selection method based on binary dandelion algorithm is proposed. • The algorithm applies a sine-cosine operator and a restart strategy. • Mutual information and quick bit mutation improve the performance of the algorithm. • The algorithm performs well on datasets with larger dimensions. Feature selection (FS) is an important data preprocessing technology for machine learning and data mining. Metaheuristic algorithm (MH) has been widely used in feature selection because of its powerful search function. This paper presents an improved Binary Dandelion Algorithm using Sine Cosine operator and Restart strategy (SCRBDA) for feature selection. First, the sine cosine operator is used in the radius formula of the core dandelions (CD), which significantly enhances the ability of algorithm development and exploration. Secondly, the algorithm uses a restart strategy to increase its ability to get rid of local optimum. Thirdly, mutual information is used to guide the generation of some dandelions, which pays more attention to the correlation between the selected features and categories. Finally, quick bit mutation is used as the mutation strategy to improve the diversity of the population. The SCRBDA proposed in this paper was tested on 18 datasets of different sizes from UCI machine learning database. The SCRBDA was compared with 8 other classical feature selection algorithms, and the performance of the proposed algorithm was evaluated through feature subset size, classification accuracy, fitness value, and F1-score. The experimental results show that SCRBDA achieves the best performance, which has stronger feature reduction ability and achieves better overall performance on most datasets. Especially on large-scale datasets, SCRBDA can obtain extremely smaller feature subsets while maintaining much higher classification accuracy, and satisfactory F1-score. [ABSTRACT FROM AUTHOR]

Identifying the abnormal samples from a data set and determining their type are two key tasks of anomaly detection. However, the existing anomaly detection algorithms are generally faced with the defects of weak generalization ability and insufficient interpretation, the core reason of which is that they cannot mine specific features for different abnormal types. In this paper, a new anomaly detection algorithm aiming at feature selection for different abnormal types is developed. Inspired by self-supervised learning, we take the stationarity of variance changes of abnormal score similarity as a pretext task and combine it with wrapped search method. Then, the features and the corresponding parameters for different abnormal types can be screened to apply to the downstream task of anomaly integration detection. To verify the efficiency of the new algorithm, we conduct two sets of experiments to compare the new algorithm with 11 classical anomaly detection and 3 clustering anomaly detection algorithms on the data sets WDBC, WPBC and Wilt from DAMI database with the evaluation measures P@n , Adj- P@n , AP, Adj-AP and AUC. The experiment results show that, both in the identification and classification on abnormal samples, all performance measures of the new algorithm are explicitly better than that of the contrastive algorithms. Also, we apply the new algorithm to the Chinese auto insurance market, and find that the results can help managers to identify the main patterns of fraudulent claims and to summarize the feature combinations of fraud behaviors. In general, the new algorithm developed in this paper has the following advantages compared with traditional algorithms: 1) It can directly capture abnormal features and realize effective recognition of abnormal types, which effectively bridge the gap between abnormal judgement and feature screening. 2) Automatic screening of abnormal features can be completed under the condition of self-updating learning optimal strategy. 3) Only a few features are extracted from all features to reveal the abnormal characteristics, which significantly improves the interpretability and generalization ability of the algorithm and its results. In a word, through the novel self-supervised design method, feature screening is skillfully integrated into the anomaly detection task, which may provide a new way for anomaly detection research. [ABSTRACT FROM AUTHOR]

High dimensional and small samples characterize gene expression data and contain a large number of genes unrelated to disease. Feature selection improves the efficiency of disease diagnosis by selecting a small number of important genes. Unfortunately, existing algorithms do not consider the correlation between features, and search algorithms tend to fall into the local optimal solution in the feature search process. To this end, this paper proposes a feature filter and group evolution hybrid feature selection algorithm (FG-HFS) for high-dimensional gene expression data. Unlike existing algorithms, we propose using spectral clustering to group redundant features into a group. Then, we propose a redundant feature filter algorithm. According to the principle of approximate Markov blanket, grouped feature groups are filtered to delete these redundant features. Among them, filtered features are evenly divided by density according to the feature exponential strategy. Most importantly, we propose using the group evolution multi-objective genetic algorithm to search the filtered feature subsets and evaluate the candidate feature subsets according to the in-group and out-group so as to select the feature subsets with the highest accuracy and the least number. Experimental results show that the average accuracy (ACC) and Matthews correlation coefficient (MCC) indexes of the selected feature subsets (FSs) by the FG-HFS algorithm on 5 gene expression datasets are 92.76% and 88.76%, respectively, which are significantly better than the existing algorithms. In addition, the FSs and ACC/FSs indexes of the FG-HFS algorithm are also better than the existing algorithms, which fully proves the superiority of the FG-HFS algorithm. More importantly, the Wilcoxon and Friedman statistical experiments results show that the feature selection effect of FG-HFS algorithm is significantly better than that of existing algorithms, no matter in pairwise comparison or multiple comparison. • We propose using spectral clustering to group the features so that the in-group feature similarity is extremely high. • We propose a redundant feature filter algorithm since existing algorithms cannot filter redundant features. • we propose using the group evolution multi-objective genetic algorithm to search the filtered feature subsets. [ABSTRACT FROM AUTHOR]

• MMODE is developed as a new hybrid gene selection method for tumor identification. • MMODE combines multi-tasking and multi-objective frameworks. • MMODE uses a new elite strategy and a new guidance strategy. • MMODE selects a few genes and achieves high classification accuracy. A key preprocessing step in tumor recognition based on microarray expression profile data and machine learning is to identify tumor marker genes. Gene selection aims to select the most relevant gene subset from the original ultra-high dimensional microarray expression profile data to improve tumor identification performance. Inspired by evolutionary multitasking (EMT) and multi-objective optimization, this paper puts forward a novel multitasking multi-objective differential evolution gene selection algorithm (MMODE) which uses new elite and guidance strategies to select the best gene subsets. MMODE initializes two different populations according to different filtering criteria to increase the diversity of the search. These two populations guide their respective populations to search in the optimal direction through knowledge transfer in the evolutionary process. In addition, MMODE employs new elite and guidance strategies that enables individuals to narrow the search range and jump out of local optima. The proposed algorithm is validated on 13 publicly available microarray expression datasets in comparison with state-of-the-art gene selection algorithms. The experimental results show that MMODE can find smaller gene subsets and achieve higher classification accuracy compared with other algorithms. [ABSTRACT FROM AUTHOR]

• A causal discovery algorithm based on rank correlation. • Apply Kendall correlation coefficient to causal discovery. • Lung nodule detection algorithm based on rank correlation. • Results demonstrate the feasibility of the proposed method. • The method can be applied to other medical tasks. Early diagnosis can significantly improve the survival rate of lung cancer patients. This study attempts to construct a causal structure network between the computational and semantic features of lung nodules through causal discovery algorithms, and to detect and prevent lung nodules based on this network. For complex and diverse lung nodule data sets, this paper proposes a new causal lung nodule detection algorithm Tau-CSFS in combination with rank correlation methods. The algorithm can effectively mine the causal relationship among lung cancer data that obey the non-linear non-Gaussian distribution, and the mixture of continuous and discrete variables, and has good predictive performance. We made three main contributions. First, we proved that the Kendall rank correlation coefficient that does not require data distribution can be used as a standard for independence test. Second, we applied Kendall rank correlation to Bayesian structure learning, and proposed a new causal discovery algorithm: Tau-CS algorithm based on hypothesis testing. The third contribution is to combine the Tau-CS algorithm with the feature selection method, and further propose the Tau-CSFS algorithm, which solves the problem of causality mining and diagnosis detection of lung nodule data. In the experiment, the Tau-CS algorithm is compared with the prior art on 7 Bayesian networks on the additive noise structure model, and it is proved that the algorithm has a better accuracy of causal structure learning. Finally, in the lung nodule detection stage, using the processed LIDC data set to perform two-classification and multi-classification experiments on seven semantic categories, the average accuracy of the Tau-CSFS algorithm reached 85.84% and 83.32%. The Tau-CSFS algorithm are better than comparable similar algorithms in the comprehensive performance index. The results show that the proposed algorithm has good detection performance and wide application prospects. [ABSTRACT FROM AUTHOR]

Optimization algorithms have shown significant advantages in solving diverse several real-world problems, especially where are limitations in computations and hardware requirements. However, they might get trapped in local optima or fast convergence. To solve these challenges, the exploration ability of an optimization algorithm should be evaluated and improved. This paper proposes a new approach for solving global optimization and feature selection problems to overcome these drawbacks and balance exploitation and exploration abilities. We propose a hybrid approach combining both, Gradient-Based Optimizer (GBO) using Slime Mould Algorithm (SMA), which exploits the strength factors of Gradient-Based Optimizer and Slime Mould Algorithm. The SMA is applied as a local search for the GBO to enhance its ability to explore more regions in the search space. This strategy is employed to achieve a balance between both exploration and exploitation. Two main experiments are applied to evaluate the performance of the GBOSMA. The first one solves a well-known global optimization problem (CEC2017), whereas the second one solves feature selection problems using several benchmark datasets. The GBOSMA is compared to the standard GBO, SMA, and the most recent optimization algorithms in both experiments. The results also show that the GBOSMA approach obtains promising results in both experiments outperforming other algorithms. Comparing with eight well-known optimization algorithms, the results (for both global optimization and feature selection problems) demonstrate that the GBOSMA overcomes others in terms of performance, speed and stability. • Improving the search ability of the GBO optimizer using the SMA algorithm. • Evaluating the performance of the proposed method using CEC2017 functions. • Selecting the most relative features to increase the classification accuracy. [ABSTRACT FROM AUTHOR]

With the rapid growth of the number of connected devices that exchange personal, sensitive, and important data through the IoT based global network, attacks that are targeting security services are increasing as well. Accordingly, there is a need for security solutions that are suitable for IoT environment. A network intrusion detection system (NIDS) is a solution that examines network traffic and alerts system administrators if there are security breaches. In this paper, an enhanced version of pigeon-inspired optimization (PIO) is proposed which enhances PIO by adding a local search algorithm named (LS-PIO). Moreover, an ensemble learning approach, which is based on multiple one-class classifiers, has been used in order to improve the performance of proposed NIDS. Four benchmark datasets were used to evaluate the LS-PIO and ensemble based NIDS which are BoT-IoT, UNSW-NB15, NLS-KDD and K D D C U P P 99 . The evaluation took into consideration F-score, accuracy, AUC, FPR and TPR. Results show that the suggested approach outperforms other NIDS techniques that are selected from state-of-the-art relevant research found in the literature. • Proposing a feature selection algorithm based on local search and pigeon optimizer. • Introducing an ensemble technique to used in IDS for IoT cyber-Attack. • Investigating Tabu and Hill Climbing search for improving PIO optimizer performance. • Anomaly detection based on one class classifiers has been explored. [ABSTRACT FROM AUTHOR]

• Proposes Tri-Staged Feature Selection (TFS) for multi-class heterogeneous datasets. • Initial features are selected using Kruskal Wallis Test. • Refinement of obtained features using Memetic Algorithm with local beam search. • Final feature set refinement using Cuckoo search algorithm for better classification. • Experiments conducted on 12 real datasets for validation of proposed method. Classification algorithms and their preprocessing operations usually performs on feature selection on homogeneous or heterogeneous attributes, binary or multi-class labels separately. Only very few methods attempt to perform feature selection on datasets with heterogeneous multi-class attributes. In order to bridge this gap with better classification performance, the paper proposes a Tri-staged Feature Selection (TFS) methodology which performs (i) Feature selection using Kruskal Wallis test (ii) Refinement of feature selection using a new Memetic Algorithm with local beam search and genetic algorithm operations and (iii) Further refinement of feature selection using Cuckoo Search algorithm. Proper tradeoff between both exploration and exploitation is maintained in the proposed method. The experimental results on 12 datasets show that the proposed method is better than that of state-of-the-art methods used for feature selection in terms of multi-class accuracy, hamming loss, ranking loss, normalized coverage and convergence rate for multi-class heterogeneous datasets. [ABSTRACT FROM AUTHOR]

Feature selection techniques have been presented to allow us to choose a small subset of the original components' relevant features by removing irrelevant or redundant features. Feature selection is essential for many reasons such as simplification, performance, computational efficiency, and quality interpretability. Owing to the importance mentioned above, many researchers have proposed and developed many algorithms to solve the feature selection problem. Although these approaches produce useful results, they possess some shortcomings like inadequate feature reduction. In this paper, a novel feature selection algorithm based on the crow search algorithm is presented. The algorithm uses dynamic awareness probability to keep the balance between the local and global search processes. Moreover, a novel neighborhood assigning strategy has been introduced to optimize the local search. Considering the best-selected features in each iteration helps attain more benefits in global search. The main superiority of the proposed algorithm is the significant feature reduction along with retaining the accuracy. Compared to enhanced crow search algorithm, the proposed algorithm has improved the feature reduction metric and fitness metric by 27.12% and 5.16%, respectively, while losing the accuracy metric by only 0.53%. Several popular UCI datasets have been employed to evaluate the proposed feature selection algorithm. The experimental results show that the proposed algorithm outperformed other feature selection algorithms in state-of-the-art related works regarding feature reduction and accuracy. • Improve the balance between the local and global search. • Introducing a new neighborhood concept for improving the local search. • Proposing a new method to search more purposeful during exploration. • Increasing convergence rate using chaos. • Being pioneer in reducing the dataset volume. [ABSTRACT FROM AUTHOR]

• 3 layer neural networks outperform other ML algorithms in automated assessments. • In subjective assignments, deep neural networks can improve score prediction. • Word embeddings and linguistic feature results in more precise text mining models. • Extraction and selection of appropriate features can improve results. Machine learning (ML) has recently gained popularity in a variety of domains for automating tasks. This has also been used in the evaluation of student responses/ assignments. This paper presents a qualitatively enhanced methodology for automated score prediction of subjective assignments. To consider all the major aspects of a human grader, 21 linguistic features related to syntactical, grammatical, sentimental, and readability are analyzed from the assignments. This study makes use of the ASAP competition dataset from Kaggle. The evaluation metric used is Quadratic Weighted Kappa (QWK), which measures the agreement between the human graded score and the predicted score. The effect of appropriate feature selection has been observed using Mutual Information Regression. Four ML algorithms are investigated on identified linguistic features. 3 Layer Neural Network with feature selection performed well among all chosen ML algorithms with average QWK of 0.678. To include the benefits of deep learning, a new hybrid model (LF-BiLSTM-att-FS) is proposed that combines a higher level deep neural network (DNN) with the selected features. Pre-trained Glove embedding is used to include contextual information of the text. The proposed model results demonstrated an improvement in overall accuracy, with an average QWK value of 0.768. [ABSTRACT FROM AUTHOR]

• GBO-kNN is a proposed framework based on a wrapper approach for features selection. • The aim of the research is to reach maximum classification accuracy. • The performance of GBO-kNN is evaluated against real benchmark datasets. • The GBO-kNN performance is compared against seven recent metaheuristic algorithms. • Results showed high effectiveness on one dataset and moderate on another. Virtual Screening (VS) in drug discovery campaigns deals with enormously large databases containing massive redundant and/or irrelevant features. Thus, the pre-selection process of virtual screening requires developing a fast and precise classification methodology for filtering such a huge database. This paper proposed a framework depending on a wrapper selection approach for features selection. It consists of 1) an optimizer: Gradient-Based Optimizer (GBO), that hybridized with 2) the classifier: k-nearest neighbor (k-NN). The performance of the introduced framework, GBO-kNN, is estimated using real-world benchmark datasets; the QSAR Biodegradation which is composed of 41 features, and the Monoamine Oxidase (MAO) that consisted of 1665 features. the GBO-kNN framework is compared against seven recent swarm intelligence algorithms: Hybrid Harris Hawks Optimization Algorithm (HHO), Grey Wolf Optimization Algorithm (GWO), Butterfly Optimization Algorithm (BOA), Dragonfly Algorithm (DA), Moth-Flame Optimization Algorithm (MFO), Sine Cosine Algorithm (SCA), and Salp Swarm Algorithm (SSA) for reaching maximum classification accuracy. The experimental results exhibited that the proposed process, in comparison with other algorithms, achieved high effective accuracy of 98.8% over the high dimensional dataset (MAO) and a moderate effective performance over the low dimensional dataset (QSAR Biodegradation). The proposed framework source code is available at https://codeocean.com/capsule/9906421/tree/v1. [ABSTRACT FROM AUTHOR]

Feature selection (FS) is recognized as one of the majority public and challenging problems in the Machine Learning domain. FS can be examined as an optimization problem that needs an effective optimizer to determine its optimal subset of more informative features. This paper proposes a wrapper FS method that combines chaotic maps (CMs) and binary Group Search Optimizer (GSO) called CGSO, which is used to solve the FS problem. In this method, five chaotic maps are incorporated with the GSO algorithm's main procedures, namely, Logistic, Piecewise, Singer, Sinusoidal, and Tent. The GSO algorithm is used as a search strategy, while k-NN is employed as an induction algorithm. The objective function is to integrate three main objectives: maximizing the classification accuracy value, minimizing the number of selected features, and minimizing the complexity of generated k-NN models. To evaluate the proposed methods' performance, twenty well-known UCI datasets are used and compared with other well-known published methods in the literature. The obtained results reveal the superiority of the proposed methods in outperforming other well-known methods, especially when using binary GSO with Tent CM. Finally, it is a beneficial method to be utilized in systems that require FS pre-processing. • A novel feature selection method is proposed for classification task. • Five Chaotic Group Search Optimizers are investigated. • Twenty UCI datasets were used in the experiments. • Our methods got a higher accuracy results using a fewer number of features. [ABSTRACT FROM AUTHOR]

A context-aware recommender system (CARS) utilizes users' context to provide personalized services. Contextual information can be derived from sensors in order to improve the accuracy of the recommendations. In this work, we focus on CARSs with high-dimensional contextual information that typically impacts the recommendation model, for example, by increasing the model's dimensionality and sparsity. Generating accurate recommendations is not enough to constitute a useful system from the user's perspective, since the use of some contextual information may cause problems, such as draining the user's battery, raising privacy concerns, and more. Previous studies suggested reducing the amount of contextual information utilized by using domain knowledge to select the most suitable information. This approach is only applicable when the set of contexts is small enough to handle and sufficient for preventing sparsity. Moreover, hand-crafted context information may not represent an optimal set of features for the recommendation process. Another approach is to compress the contextual information into a denser latent space, but this may limit the ability to explain the recommended items to the users or compromise their trust. In this paper, we present a multi-step approach for selecting low-dimensional subsets of contextual information and incorporating them explicitly within CARSs. At the core of our approach is a novel feature selection algorithm based on genetic algorithms, which outperforms state-of-the-art dimensionality reduction CARS algorithms by improving recommendation accuracy and interpretability. Over the course of evolution, thousands of diverse feature subsets are generated; a deep context-aware model is produced for each feature subset, and the subsets are stacked together. The resulting stacked model is accurate and only uses interpretable, explicit features. Our approach includes a mechanism of tuning the different underlying algorithms that affect user concerns, such as privacy and battery consumption. We evaluated our approach on two high-dimensional context-aware datasets derived from smartphones. An empirical analysis of our results confirms that our proposed approach outperforms state-of-the-art CARS models while improving transparency and interpretability for the user. In addition to the empirical results, we present several use cases, examples and methodology of how researchers, domain experts and CARS modelers can tweak the feature selection algorithm to improve various user concerns and interpretability. • A novel genetic algorithm-based approach for context-aware recommender systems. • A means of addressing user concerns, such as battery usage, privacy, etc. • Accurate and interpretable models that take users' concerns into account. • State-of-the-art qualitative and quantitative results on high-dimensional datasets. [ABSTRACT FROM AUTHOR]

Link prediction refers to predicting the connection probability between two nodes in terms of existing observable network information, such as network structural topology and node properties. Although traditional similarity-based methods are simple and efficient, their generalization performance varies widely in different networks. In this paper, we propose a novel link prediction approach ICP based on inductive matrix completion, which recoveries node connection probability matrix by applying node features to a low-rank matrix. The approach first explores a comprehensive node feature representation by combining different structural topology information with node importance properties via feature construction and selection. The selected node features are then used as the input of a supervised learning task for solving the low-rank matrix. The node connection probability matrix is finally recovered by a bi-linear function, which predicts the connection probability between two nodes with their features and the low-rank matrix. In order to demonstrate the ICP superiority, we took eleven related efforts including two recent methods proposed in 2020 as baseline methods, and it is shown that ICP has stable performance and good universality in twelve different real networks. Compared with the baseline methods, the improvements of ICP in terms of the average AUC results are ranging from 3.81% ∼ 12.77% and its AUC performance is improved by 0.08% ∼ 3.54% compared with the best baseline method. The limitation of ICP lies in its high computational complexity due to the feature construction, but the complexity can be reduced by replacing complex features with node semantic attributes if there are additional data available. Moreover, it provides a potential link prediction solution for large-scale networks, since inductive matrix completion is a supervised learning task, in which the underlying low-rank matrix can be solved by representative nodes instead of all their nodes. • A novel link prediction approach based on inductive matrix completion is proposed. • A comprehensive node feature representation is employed. • Link prediction is based on the node features and the factorized matrices. • Comparisons have been done with both classical and state-of-the-art methods. • The approach provides a potential link prediction solution for large-scale networks. [ABSTRACT FROM AUTHOR]

Silent diseases is an umbrella term that captures a spectrum of chronic illnesses that produce no clinically obvious signs and are diagnosed at advanced stages when the damage is irreversible. Current diagnostic strategies of silent diseases depend on self-reported symptoms and observed behavior through extended periods of time, and until now there are no specific clinical tests to diagnose silent diseases. Scientific research suggests the importance of early diagnosis to restore the functionality and reduce diseases-related complications. Previous studies primarily focused on feature selection methods to aid in medical diagnosis. Traditional feature selection methods are primarily focused on correct classification and often ignore features' costs; the cost of clinical tests required to acquire the feature value. However, in medical diagnosis, features have different associated costs. Because ignoring features' costs may result in a high cost diagnostic strategy that cannot be used in practice, developing a low-cost diagnostic strategy remains a subject of much interest. In this paper, new Mixed Integer Programming (MIP) models, namely, Cost-sensitive Support Vector Machine (CS-SVM) and Cost-sensitive Multi-surface Method Tree (CS-MSMT) that allow for simultaneous selection of low-cost and informative features are proposed. The CS-SVM and CS-MSMT are superior because they have the ability to account for shared costs. The CS-SVM and CS-MSMT were modified to embed shared costs across feature groups, and are termed Discounted CS-SVM (d CS-SVM) and Discounted CS-MSMT (d CS-MSMT), respectively. Computationally effective algorithm that integrates aggressive bound tightening with the MIP formulation is proposed. To demonstrate the effectiveness of the proposed models, different analysis paradigms are conducted on six UCI medical datasets; Chronic Kidney Disease, Hepatitis, Heart Disease, Thyroid, Diabetes and Leukemia. The results demonstrate the efficiency and robustness of the CS-SVM and CS-MSMT (and consequently the d CS-SVM and d CS-MSMT) under various conditions. The CS-SVM and CS-MSMT improved accuracy by 10.3% and 3.4% and reduced costs by 94.3% and 72.4% in the leukemia dataset, respectively. • New MIP models for cost-sensitive feature selection are proposed. • The models are robust enough to account for shared cost across feature groups. • Aggressive bound tightening within Branch and Cut algorithm was used. • The proposed cost-sensitive feature selection models outperformed existing feature selection techniques. • The models improved the accuracy up by 10.3% and decreased the cost up to 96%. [ABSTRACT FROM AUTHOR]

• Enhance EO algorithm using ReliefF algorithm and the local search strategy. • Propose new feature selection method based on a hybridization RBEO-LS method. • Evaluate RBEO-LS using 16 UCI datasets and 10 high dimensional biological datasets. • Results show superiority of RBEO-LS among other state-of-the-art methods. Feature selection (FS) is an important task in any classification process and aims to choose the smallest features number that yields higher classification accuracy. FS can be formulated as a combinatorial NP-hard problem for which robust metaheuristics are used as efficient wrapper-based FS approaches. However, when applied for high dimensional datasets that present large features number and few samples, the effectiveness of such wrapper-metaheuristics degraded, and their computation costs increased. To tackle this problem, we propose in this paper a hybrid FS approach based on the ReliefF filter method and a novel metaheuristic Equilibrium Optimizer (EO). The proposed method, called RBEO-LS, is composed of two phases. In the first phase, the ReliefF algorithm is used as a preprocessing step to assign weights for features, which estimate their relevance to the classification task. In the second phase, the binary EO (BEO) is used as a wrapper search approach. The features are ranked according to their weights and are used for the initialization of the BEO population. We embedded the BEO with a local search strategy to improve its performance by adding relevant features and removing redundant ones from the features subset guided by the features ranking and the Pearson coefficient correlation. The performance of the developed algorithm has been evaluated on sixteen UCI datasets and ten high dimensional biological datasets. The UCI datasets contain a high number of samples and a small or medium number of features. The biological datasets present a high number of features with few samples. The results demonstrate that the use of the ReliefF algorithm and the local search strategy improves the performance of the EO algorithm. The results also show the superiority of the RBEO-LS, among other state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

• FCS is proposed as a complementary step for principal component analysis. • PCA-FCS identifies the most relevant components based on their level of contribution. • PCA-FCS simultaneously performs dimension reduction and component selection. • PCA-FCS has substantially improved the accuracy of the random forest algorithm. Principal component analysis (PCA) is one the most effective and widely used dimensionality-reduction techniques which aggregates the maximal variance in the first few components. In many applications, the first few components of PCA are used in place of the original variables in statistical and machine learning models; nevertheless, such use does not guarantee the selection of the most relevant components to the target variable. This paper presents an efficient approach in which all components, rather than simply the first few, are considered as input for the random forest (RF) model, and a feature selection algorithm is integrated with the RF to select the most relevant components, hence called fully component selection (FCS). The proposed method was evaluated on spectroscopic data comprising 70 soil samples with 2050 spectral bands to estimate soil organic carbon (SOC). The FCS approach was compared to the RF model once considering the original features, then again using only the first few components, and again using all components without feature selection. The results showed that RF-FCS substantially outperformed other approaches, such that the R2 was increased between 25.2% and 55.5%. Furthermore, the findings indicated that the most relevant principal components (PCs) to the target variable were not the first few, but PC6, PC7, PC8, PC15, and PC43. Using the FCS approach increased model performance substantively, and coupling it with machine learning and statistical models is strongly recommended for high dimensional data applications. [ABSTRACT FROM AUTHOR]

Feature selection in real-world data mining problems is essential to make the learning task efficient and more accurate. Identifying the best feature selection algorithm, among the many available, is a complex activity that still relies heavily on human experts or some random trial-and-error procedure. Thus, the automated machine learning community has taken some steps towards the automation of this process. In this paper, we address the metalearning challenge of recommending feature selection algorithms by proposing a novel meta-feature engineering model. Our model considers a broad collection of meta-features that enable the study of the relationship between the dataset properties and the feature selection algorithm performance in terms of several criteria. We arrange the input meta-features into eight categories: (i) simple, (ii) statistical, (iii) information-theoretical, (iv) complexity, (v) landmarking, (vi) based on symbolic models, (vii) based on images, and (viii) based on complex networks (graphs). The target meta-features emerge from a multi-criteria performance measure, based on five individual performance indexes, that assesses feature selection methods grounded in information, distance, dependence, consistency, and precision measures. We evaluate our proposal using a recently developed framework that extracts the input meta-features from 213 benchmark datasets, and ranks the assessed feature selection algorithms, to fill in the target meta-features in meta-bases. This evaluation uses five state-of-the-art classification methods to induce recommendation models from meta-bases: C4.5, Random Forest, XGBoost, ANN, and SVM. The results showed that it is possible to reach an average accuracy of up to 90% applying our meta-feature engineering model. This work is the first to use an extensive empirical evaluation to provide a careful discussion of the strengths and limitations of more than 160 meta-features. These meta-features, while designed to aid the task of feature selection algorithm recommendation, can readily be employed in other metalearning scenarios. Therefore, we believe our findings are a valuable contribution to the fields of automated machine learning and data mining, as well as to the feature extraction and pattern recognition communities. • A novel meta-feature engineering model recommends feature selection algorithms. • The proposal obtains promising results from 213 datasets with hit rates of up to 90%. • Some simple, landmarking, image, and graph-based input meta-features highlighted. • A multi-criteria performance measure rigorously assesses candidate algorithms. • Chains of binary or multiclass classifiers can efficiently rank candidate algorithms. [ABSTRACT FROM AUTHOR]

• A novel multi-objective feature selection algorithm is proposed. • It hybridizes filter and wrapper models into whale optimization algorithm. • It has been validated through ten datasets and compared against five algorithms. • Yields solutions with fewer features and improves the classification accuracy. Feature selection aims at finding the minimum number of features that result in high classification accuracy. Accordingly, the feature selection is considered as a multi-objective problem. However, most existing approaches treat feature selection as single-objective problem, and they are often divided into two main categories: filter and wrapper methods. Filters are known as fast methods but less accurate, while wrappers are computationally expensive but with high classification performance. This paper proposes a novel hybrid filter-wrapper feature selection approach using whale optimization algorithm (WOA). The proposed method is a multi-objective algorithm in which a filter and wrapper fitness functions are optimized simultaneously. Our algorithm's efficiency is demonstrated through an extensive comparison with seven well-known algorithms on twelve benchmark datasets. Experimental results show the ability of the proposed algorithm to obtain several subsets that include smaller number of features with excellent classification accuracy. [ABSTRACT FROM AUTHOR]

• Efficient and technically sound feature selection based on Sobol decomposition. • The proposed method takes into account interactions between features. • Superior performance vis-a-vis other state of the art sampling techniques. • The proposed method is built on solid mathematical framework. Existing feature selection methods fail to properly account for interactions between features. In this paper, we attempt to remedy this issue by using orthogonal variance decomposition to evaluate features. The orthogonality of the decomposition allows us to directly calculate the total contribution of each feature to the output variance. As a result, we obtain an efficient and technically sound feature selection algorithm which takes into account feature interactions. The proposed algorithm has low computational complexity compared to other methods used in the literature. Numerical experiments demonstrate that our method accurately identifies relevant features and improves the accuracy of numerical models. [ABSTRACT FROM AUTHOR]

• We have designed a method for ensemble feature selection. • We model the feature selection process to a Pareto-based optimization problem. • The crowding distance between solutions is the secondary measure. • The method is an ensemble of relevancy and redundancy methods. • The proposed PEFS method outperforms competitive algorithms. In this paper, ensemble feature selection is modeled as a bi-objective optimization problem regarding features' relevancy and redundancy degree. The proposed method, which is called PEFS, first uses the modeled bi-objective optimization problem to find the non-dominated features based on the decision matrix constructed by different feature selection algorithms. In the second step, the found non-dominated features are sorted using the crowding distance in the bi-objective space. These sorted features remove from the feature space, and the process of finding the non-dominated features will continue until all the features are sorted. To illustrate the optimality and efficiency of the proposed method, we have compared our approach with some ensemble feature selection methods and basic algorithms used in the ensemble process. The results show that our method in terms of accuracy and F-score is superior to other similar methods and performs in a short running-time. [ABSTRACT FROM AUTHOR]