Search

Intrusion Detection System (IDS) can ensure the network security by identifying network intrusions according to the abnormal traffic data. However, the intrusion detection data has the problem of high dimensionality and changes with network and attack environments, which leads to the poor performance and poor portability of intrusion detection algorithms. Therefore, this paper proposes an intrusion detection algorithm based on joint symmetric uncertainty and hyperparameter optimized fusion neural network. Firstly, a feature selection method based on symmetric uncertainty and approximate Markov blanket is proposed, which fully considers the correlation and redundancy of features, and also the correlation between combined features and the class label, so as to reduce the data dimensionality. Secondly, the CNN-LSTM classifier fused with Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) is used to extract the spatial features and temporal features to improve the classification performance. Finally, the Particle Swarm Optimization (PSO) algorithm is improved and used to automatically optimize the hyperparameters of the classifier, so that the classifier can be applied to different intrusion detection datasets with better generalization ability and portability. Experiments have verified the effectiveness and superiority of the proposed algorithm on multiple evaluation indicators. • An effective algorithm for intrusion detection is proposed in this paper. • Feature selection is based on symmetric uncertainty and approximate Markov blanket. • A fusion neural network is constructed to extract the spatial and temporal features. • The PSO algorithm is improved to automatically optimize the hyperparameters. [ABSTRACT FROM AUTHOR]

• A feature selection method based on binary dandelion algorithm is proposed. • The algorithm applies a sine-cosine operator and a restart strategy. • Mutual information and quick bit mutation improve the performance of the algorithm. • The algorithm performs well on datasets with larger dimensions. Feature selection (FS) is an important data preprocessing technology for machine learning and data mining. Metaheuristic algorithm (MH) has been widely used in feature selection because of its powerful search function. This paper presents an improved Binary Dandelion Algorithm using Sine Cosine operator and Restart strategy (SCRBDA) for feature selection. First, the sine cosine operator is used in the radius formula of the core dandelions (CD), which significantly enhances the ability of algorithm development and exploration. Secondly, the algorithm uses a restart strategy to increase its ability to get rid of local optimum. Thirdly, mutual information is used to guide the generation of some dandelions, which pays more attention to the correlation between the selected features and categories. Finally, quick bit mutation is used as the mutation strategy to improve the diversity of the population. The SCRBDA proposed in this paper was tested on 18 datasets of different sizes from UCI machine learning database. The SCRBDA was compared with 8 other classical feature selection algorithms, and the performance of the proposed algorithm was evaluated through feature subset size, classification accuracy, fitness value, and F1-score. The experimental results show that SCRBDA achieves the best performance, which has stronger feature reduction ability and achieves better overall performance on most datasets. Especially on large-scale datasets, SCRBDA can obtain extremely smaller feature subsets while maintaining much higher classification accuracy, and satisfactory F1-score. [ABSTRACT FROM AUTHOR]

Identifying the abnormal samples from a data set and determining their type are two key tasks of anomaly detection. However, the existing anomaly detection algorithms are generally faced with the defects of weak generalization ability and insufficient interpretation, the core reason of which is that they cannot mine specific features for different abnormal types. In this paper, a new anomaly detection algorithm aiming at feature selection for different abnormal types is developed. Inspired by self-supervised learning, we take the stationarity of variance changes of abnormal score similarity as a pretext task and combine it with wrapped search method. Then, the features and the corresponding parameters for different abnormal types can be screened to apply to the downstream task of anomaly integration detection. To verify the efficiency of the new algorithm, we conduct two sets of experiments to compare the new algorithm with 11 classical anomaly detection and 3 clustering anomaly detection algorithms on the data sets WDBC, WPBC and Wilt from DAMI database with the evaluation measures P@n , Adj- P@n , AP, Adj-AP and AUC. The experiment results show that, both in the identification and classification on abnormal samples, all performance measures of the new algorithm are explicitly better than that of the contrastive algorithms. Also, we apply the new algorithm to the Chinese auto insurance market, and find that the results can help managers to identify the main patterns of fraudulent claims and to summarize the feature combinations of fraud behaviors. In general, the new algorithm developed in this paper has the following advantages compared with traditional algorithms: 1) It can directly capture abnormal features and realize effective recognition of abnormal types, which effectively bridge the gap between abnormal judgement and feature screening. 2) Automatic screening of abnormal features can be completed under the condition of self-updating learning optimal strategy. 3) Only a few features are extracted from all features to reveal the abnormal characteristics, which significantly improves the interpretability and generalization ability of the algorithm and its results. In a word, through the novel self-supervised design method, feature screening is skillfully integrated into the anomaly detection task, which may provide a new way for anomaly detection research. [ABSTRACT FROM AUTHOR]

High dimensional and small samples characterize gene expression data and contain a large number of genes unrelated to disease. Feature selection improves the efficiency of disease diagnosis by selecting a small number of important genes. Unfortunately, existing algorithms do not consider the correlation between features, and search algorithms tend to fall into the local optimal solution in the feature search process. To this end, this paper proposes a feature filter and group evolution hybrid feature selection algorithm (FG-HFS) for high-dimensional gene expression data. Unlike existing algorithms, we propose using spectral clustering to group redundant features into a group. Then, we propose a redundant feature filter algorithm. According to the principle of approximate Markov blanket, grouped feature groups are filtered to delete these redundant features. Among them, filtered features are evenly divided by density according to the feature exponential strategy. Most importantly, we propose using the group evolution multi-objective genetic algorithm to search the filtered feature subsets and evaluate the candidate feature subsets according to the in-group and out-group so as to select the feature subsets with the highest accuracy and the least number. Experimental results show that the average accuracy (ACC) and Matthews correlation coefficient (MCC) indexes of the selected feature subsets (FSs) by the FG-HFS algorithm on 5 gene expression datasets are 92.76% and 88.76%, respectively, which are significantly better than the existing algorithms. In addition, the FSs and ACC/FSs indexes of the FG-HFS algorithm are also better than the existing algorithms, which fully proves the superiority of the FG-HFS algorithm. More importantly, the Wilcoxon and Friedman statistical experiments results show that the feature selection effect of FG-HFS algorithm is significantly better than that of existing algorithms, no matter in pairwise comparison or multiple comparison. • We propose using spectral clustering to group the features so that the in-group feature similarity is extremely high. • We propose a redundant feature filter algorithm since existing algorithms cannot filter redundant features. • we propose using the group evolution multi-objective genetic algorithm to search the filtered feature subsets. [ABSTRACT FROM AUTHOR]

• MMODE is developed as a new hybrid gene selection method for tumor identification. • MMODE combines multi-tasking and multi-objective frameworks. • MMODE uses a new elite strategy and a new guidance strategy. • MMODE selects a few genes and achieves high classification accuracy. A key preprocessing step in tumor recognition based on microarray expression profile data and machine learning is to identify tumor marker genes. Gene selection aims to select the most relevant gene subset from the original ultra-high dimensional microarray expression profile data to improve tumor identification performance. Inspired by evolutionary multitasking (EMT) and multi-objective optimization, this paper puts forward a novel multitasking multi-objective differential evolution gene selection algorithm (MMODE) which uses new elite and guidance strategies to select the best gene subsets. MMODE initializes two different populations according to different filtering criteria to increase the diversity of the search. These two populations guide their respective populations to search in the optimal direction through knowledge transfer in the evolutionary process. In addition, MMODE employs new elite and guidance strategies that enables individuals to narrow the search range and jump out of local optima. The proposed algorithm is validated on 13 publicly available microarray expression datasets in comparison with state-of-the-art gene selection algorithms. The experimental results show that MMODE can find smaller gene subsets and achieve higher classification accuracy compared with other algorithms. [ABSTRACT FROM AUTHOR]

In most data dimensionality reduction tasks, feature selection (FS) is an essential preprocessing stage. The slime mould algorithm (SMA) is a novel metaheuristic optimization algorithm. This paper proposes a feature selection method for banknote dirtiness recognition based on mathematical functions driven slime mould algorithm. This method is applied to the feature selection of banknote dirtiness recognition, and the Lévy flight operator is used to replace the control parameter vc in SMA, and a series of mathematical functions are added to replace the control parameter vb in the slime mould algorithm to enhance the global search capability. Test results show that this method achieves the highest accuracy of 76.85 % in recognizing the banknote dirtiness. Additionally, by applying the same method to feature selection and recognition of dirtiness for the left middle white area images of banknotes, the highest accuracy reaches 74.99 %. This suggests that the left middle white area can essentially replace the whole banknote. To verify the performance of the proposed feature selection method, comparative tests with four other optimization algorithms are carried out, and the results demonstrate that three improvement strategies can effectively improve the performance of SMA while maintaining a balance between exploration and exploitation. Tests have revealed that these three algorithms have significant advantages over the past SMA in terms of recognition accuracy and convergence speed when it comes to identifying the dirtiness of banknotes. Therefore, this method is considered to be one of the most effective methods to solve the feature selection problem of banknote dirtiness. [ABSTRACT FROM AUTHOR]

The challenge of high-dimensional feature selection (FS) lies in the search technique, which needs to consider both minimizing the size of feature subset and maximizing the classification accuracy. Recently, multi-objective evolutionary algorithms (MOEAs) have shown excellent performance in solving FS tasks. However, most existing MOEAs struggle to effectively balance these two conflicting objectives when solving high-dimensional FS tasks. To cover this issue, in this paper, a decomposition-based multi-population evolutionary algorithm is proposed, called MPEA-FS. In the initialization stage, a multi-population generation strategy (MGS) based on feature weights (MGS) was adopted, with each population corresponding to a search subspace, which can improve the algorithm's search ability. During the evolutionary stage, an external population is designed to integrate the excellent feature subsets of multiple populations to achieve knowledge sharing between them. In addition, to reduce the feature dimension, a feature reduction strategy (FRS) is proposed, which can remove unimportant features while maintaining classification accuracy. Extensive experiments are performed on 13 high-dimensional datasets, and the proposed algorithm is compared with 5 state-of-the-art FS methods proposed in the last 3 years on multiple metrics. The experimental results indicate that MPEA-FS can achieve higher classification accuracy on most datasets, and the number of selected features is also competitive. • A decomposition-based multi-population feature selection algorithm is proposed. • The generation of multi-population is based on a novel feature weight method. • Multi-population knowledge sharing strategy based on elite population is proposed. • Evolution operator based on guide vector can correct the feature subset effectively. • Obtained higher classification accuracy and competitive number of features. [ABSTRACT FROM AUTHOR]

Highlights • Fault prediction improves the effectiveness of software quality assurance activities. • This paper focuses on building an effective fault prediction classifier. • Fault prediction model using Iterated feature selection algorithms with L-RNN. • We perform experiments on 19 open source projects. • Fault prediction model is best suitable for projects with faulty classes less than the threshold value. Abstract Software fault prediction (SFP) is typically used to predict faults in software components. Machine learning techniques (e.g., classification) are widely used to tackle this problem. With the availability of the huge amount of data that can be obtained from mining software historical repositories, it becomes possible to have some features (metrics) that are not correlated with the faults, which consequently mislead the learning algorithm and thus decrease its performance. One possible solution to eliminate those metrics is Feature Selection (FS). In this paper, a novel FS approach is proposed to enhance the performance of a layered recurrent neural network (L-RNN), which is used as a classification technique for the SFP problem. Three different wrapper FS algorithms (i.e, Binary Genetic Algorithm (BGA), Binary Particle Swarm Optimization (BPSO), and Binary Ant Colony Optimization (BACO)) were employed iteratively. To assess the performance of the proposed approach, 19 real-world software projects from PROMISE repository are investigated and the experimental results are discussed. Receiver operating characteristic - area under the curve (ROC-AUC) is used as a performance measure. The results are compared with other state-of-art approaches including Naïve Bayes (NB), Artificial Neural Network (ANN), logistic regression (LR), the k-nearest neighbors (k-NN) and C4.5 decision trees, in terms of area under the curve (AUC). Our results have demonstrated that the proposed approach can outperform other existing methods. [ABSTRACT FROM AUTHOR]

Highlights • Feature selection over high-dimensional colon cancer Microarray Datasets. • Features selected from both Gene Expression and DNA-Methylation Microarray datasets. • Resultant six biomarker genes for colon cancer validated using Enrichment Analysis. • Biomarker genes validated on independent datasets with 99.9% classification accuracy. Abstract Cancer is a dangerous disease that causes death worldwide. Discovering few genes relevant to one cancer disease can result in effective treatments. The challenge associated with the Microarray datasets is its high dimensionality; the huge number of features compared to the modest number of samples in these datasets. Recent research efforts attempted to reduce this high-dimensionality using different feature selection techniques. This paper presents an ensemble feature selection technique based on t -test and genetic algorithm. After preprocessing the data using t -test, a Nested Genetic Algorithm, namely Nested-GA, is used to get the optimal subset of features by combining data from two different datasets. Nested-GA consists of two Nested Genetic Algorithms (outer and inner) that run on two different kinds of datasets. The Outer Genetic Algorithm (OGA-SVM) works on Microarray gene expression datasets, whereas the Inner Genetic Algorithm (IGA-NNW) runs on DNA Methylation datasets. Nested-GA is performed on a colon cancer dataset with 5-fold cross validation. After applying Nested-GA, the Incremental Feature Selection (IFS) strategy is used to get the smallest optimal genes subset. The genes subset has been validated on an independent dataset resulting in 99.9% classification accuracy. Consequently, the biological significance of the resulting optimal genes is validated using Enrichment Analysis. Moreover, the results of Nested-GA have been compared to the results of other feature selection algorithms that have been run on either Gene Expression or DNA Methylation datasets. From the experimental results, Nested-GA showed the highest classification performance with a small optimal feature subset compared to the other algorithms. Furthermore, by running Nested-GA on lung cancer datasets that contain two different cancer subtypes, it resulted in significantly better classification accuracy (98.4%) compared to the accuracy of a previous research (84.6%) that utilized lung cancer DNA-Methylation data only. [ABSTRACT FROM AUTHOR]

Highlights • Two binary variants of the butterfly optimization algorithm (BOA) are proposed. • Two transfer functions are used to map the continuous search space to discrete one. • Twenty one UCI datasets are utilized in the experiments. • A superior performance of the proposed binary variants is proved in the experiments. Abstract In this paper, binary variants of the Butterfly Optimization Algorithm (BOA) are proposed and used to select the optimal feature subset for classification purposes in a wrapper-mode. BOA is a recently proposed algorithm that has not been systematically applied to feature selection problems yet. BOA can efficiently explore the feature space for optimal or near-optimal feature subset minimizing a given fitness function. The two proposed binary variants of BOA are applied to select the optimal feature combination that maximizes classification accuracy while minimizing the number of selected features. In these variants, the native BOA is utilized while its continuous steps are bounded in a threshold using a suitable threshold function after squashing them. The proposed binary algorithms are compared with five state-of-the-art approaches and four latest high performing optimization algorithms. A number of assessment indicators are utilized to properly assess and compare the performance of these algorithms over 21 datasets from the UCI repository. The experimental results confirm the efficiency of the proposed approaches in improving the classification accuracy compared to other wrapper-based algorithms, which proves the ability of BOA algorithm in searching the feature space and selecting the most informative attributes for classification tasks. [ABSTRACT FROM AUTHOR]

Background: Classification and localization of focal epileptic seizures provide a proper diagnostic procedure for epilepsy patients. Visual identification of seizure activity from long-term electroencephalography (EEG) is tedious, time-consuming and leads to human error. Therefore, there is a need for an automated classification system. Methods: In this paper, we introduce a tool called CADFES: computerized automated detection of focal epileptic seizures. For the study, total 41.66 hours of EEG data from the Bern-Barcelona database was used. Set of 28 features were extracted from time, frequency, and statistical domain and significant features were selected using neighborhood component analysis (NCA). In NCA, optimization of regularization parameter ensured better classification accuracy (less classification loss) with seven features. The performance of the algorithm was assessed using support vector machine (SVM), K-nearest neighbor (K-NN), random forest and adaptive boosting (AdaBoost) classifiers. Results: Experimental results revealed sensitivity, specificity, accuracy, positive predictive rate, negative predictive rate, and area under the curve of 97.6%, 94.4%, 96.1%, 92.9%, 98.8% and 0.96 respectively using the SVM classifier. Finally, MATLAB based software tool referred to as CADFES was introduced for automated classification of focal and non-focal seizures. Comparison results ensure that proposed study is superior to existing methods. Hence, it is expected to perform better at the hospitals for automated classification of focal epileptic seizures in real-time. [ABSTRACT FROM AUTHOR]

In recent years, cyber-attacks against businesses and organizations have increased. Such attacks usually result in significant damage to the organization, such as the loss and/or leakage of sensitive and confidential information. Because email communication is an integral part of daily business operations, attackers frequently leverage email as an attack vector in order to initially penetrate the targeted organization. Email message allows the attacker to deliver dangerous content to the victim, such as malicious attachments or links to malicious websites. Existing email analysis solutions analyze only specific parts of the email using rule-based methods, while other important parts remain unanalyzed. Existing anti-virus engines primarily use signature-based detection methods, and therefore are insufficient for detecting new unknown malicious emails. Machine learning methods have been shown to be effective at detecting maliciousness in various domains and particularly in email. Previous works which used machine learning methods suggested sets of features which offer a limited perspective over the whole email message. In this paper, we propose a novel set of general descriptive features extracted from all email components (header, body, and attachments) for enhanced detection of malicious emails using machine learning methods. The proposed features are extracted just from the email itself; therefore, our features are independent, since the extraction process does not require an Internet connection or the use of external services or other tools, thereby meeting the needs of real-time detection systems. We conducted an extensive evaluation of our new novel features against sets of features suggested by previous academic work using a collection of 33,142 emails which contains 38.73% malicious and 61.27% benign emails. The results show that malicious emails can be detected effectively when using our novel features with machine learning algorithms. Moreover, our novel features enhance the detection of malicious emails when used in conjunction with features suggested by related work. The Random Forest classifier achieved the highest detection rates, with an AUC of 0.929, true positive rate (TPR) of 0.947, and false positive rate (FPR) of 0.03. We also present the IDR (integrated detection rate), a new measure which helps calibrate the threshold of a machine learning classifier in order to achieve the optimal TP and FP rates, which are the most important measures for a real-time and practical cyber-security application. [ABSTRACT FROM AUTHOR]

Software Defect Prediction (SDP) plays a vital role in the software development life cycle as it helps identify and fix software defects. However, predicting software defects with irrelevant features and overlapping classes is challenging and can lead to lengthy training and low model accuracy. To address these challenges, this research introduces a novel Depth Linear Discrimination-Oriented Feature Selection Method based on Adaptive Sine Cosine Algorithm, named Depth Adaptive Sine Cosine Feature Selection (DASC-FS). DASC-FS integrates the Adaptive Sine Cosine Algorithm (ASCA) as a search algorithm to determine the relevant features and adopts Depth Linear Discriminant Analysis (D-LDA) to identify the discriminative features that maximize class separation. The paper proposes ASCA which is a metaheuristic algorithm meticulously designed to enhance the search capabilities of the standard Sine Cosine Algorithm (SCA). Combining the simplicity of the SCA with the efficiency of multiple mutation operators inspired by Genetic Algorithms (GA), ASCA enhances the diversity of the solutions and imparts remarkable adaptability to various situations. Furthermore, this study introduces a novel linear discriminant method, called Depth Linear Discriminant Analysis (D-LDA) to enhance the robustness of the original LDA. D-LDA systematically integrates the matrix depth concept into LDA, offering a systematic approach to address the challenges associated with scatter matrix estimation. As matrix depth measures how central or deep a particular matrix is within a distribution with respect to different directions, it is an efficient tool for computing a robust scatter matrix estimator that can handle outliers and complex data structures. The experimental results showed that DASC-FS consistently obtains the highest accuracy compared to most existing methods by integrating ASCA and D-LDA, thereby considering both accuracy optimization and class separation. The results also show that the use of multiple mutation operators in ASCA improves the search process capabilities. The results also show that the capacity of D-LDA to reduce data dimensionality and increase class separation yields highly competitive results compared to other LDAs. Finally, features related to code size and complexity have emerged as key factors for SDP because they consistently rank as important features across different classifiers and datasets. DASC-FS offers a valuable solution in domain knowledge for enhancing predictive accuracy and understanding factors contributing to software defects through enhanced search capabilities, robust scatter matrix estimation, and the ability to reduce data dimensionality. [ABSTRACT FROM AUTHOR]

Dimensionality reduction becomes a significant problem due to the proliferation of high dimensional data. Sparse preserving projection (SPP) obtains the intrinsic geometric structure of the data, which contains natural discriminating information, and avoids the selection of parameters as well. However, SPP neglects the global structures since it computes the sparse representation of each data individually. Low rank representation (LRR), another commonly used dimensionality reduction method, finds the lowest rank representation of all data jointly, and is capable of capturing the global structures of data. Therefore in this paper, we propose a method, global and intrinsic geometric structure embedding for unsupervised feature selection (GGEFS), by constructing a low-rank-sparse graph. Our GGEFS method contains the loss of information, the preservation of structural information and the sparse regularization of projection matrix, on which we impose l 2,1/2 -matrix norm to select sparser and discriminative features. An effective iterative algorithm based on Lagrange Multiplier method is described to solve GGEFS. Extensive experimental results demonstrate that the proposed algorithm outperform several state-of-the-art unsupervised feature selection methods. [ABSTRACT FROM AUTHOR]

This paper presents new feature selection algorithms for aggregate data analysis. Data aggregation is commonly used when it is not appropriate to model the relationship between a response and explanatory variables at an individual-level. We investigate substantial challenges in analysis for aggregate data. Then, we propose a groupwise feature selection method that addresses (i) the change in dataset depending on the selection of predictor variables, (ii) the presence of potential missing responses, and (iii) the suitability of model selection criteria when comparing models using different datasets. In application to real auto insurance data, we find a set of important predictors to classify the policyholders into some homogeneous risk groups. Our results clearly demonstrate the potential of the proposed feature selection method for aggregate data analysis in terms of flexibility and computational complexity. We expect that the proposed algorithms would be further applied into a wide range of decision-making tasks using aggregate data as they are applicable to any type of data. [ABSTRACT FROM AUTHOR]

By present, many models of bankruptcy forecasting have been developed, but this area remains a field of research activity; little is known about the practical application of existing models. In our opinion, this is because the use of existing models is limited by the conditions in which they are developed. Another question concerns the factors that can be significant for forecasting. Many authors suggest that indicators of the external environment, corporate governance as well as firm size contain important information; on the other hand, the large number of factors does not necessary increase predictive ability of a model. In this paper, we suggest the genetic algorithm based two-step classification method (TSCM) that allows both selecting the relevant factors and adapting the model itself to application. Classifiers of various models are trained at the first step and combined into the voting ensemble at the second step. The combination of random sampling and feature selection techniques were used to ensure the necessary diversity level of classifiers at the first step. The genetic algorithms are applied at the step of features selection and then at the step of weights determination in ensemble. The characteristics of the proposed method have been tested on the balanced set of data. It included 912 observations of Russian companies (456 bankrupts and 456 successful) and 55 features (financial ratios and macro/micro business environment factors). The proposed method has shown the best accuracy (0.934) value among tested models. It has also shown the most balanced precision-recall ratio. It found bankrupts (recall = 0.953) and not bankrupts (precision = 0.910) rather accurately than other tested models. The ability of method to select the task-relevant features has been also tested. Excluding the features that are significant for less than 50% of the classifiers in the ensemble improved the all performance metrics (accuracy = 0.951, precision = 0.932, recall = 0.965). So, the proposed method allows to improve the advantages and alleviate the weaknesses inherent in ordinary classifiers, enabling the business decisions support with a higher reliability. [ABSTRACT FROM AUTHOR]

Feature selection, both for supervised as well as for unsupervised classification is a relevant problem pursued by researchers for decades. There are multiple benchmark algorithms based on filter, wrapper and hybrid methods. These algorithms adopt different techniques which vary from traditional search-based techniques to more advanced nature inspired algorithm based techniques. In this paper, a hybrid feature selection algorithm using graph-based technique has been proposed. The proposed algorithm has used the concept of Feature Association Map (FAM) as an underlying foundation. It has used graph-theoretic principles of minimal vertex cover and maximal independent set to derive feature subset. This algorithm applies to both supervised and unsupervised classification. The performance of the proposed algorithm has been compared with several benchmark supervised and unsupervised feature selection algorithms and found to be better than them. Also, the proposed algorithm is less computationally expensive and hence has taken less execution time for the publicly available datasets used in the experiments, which include high-dimensional datasets. [ABSTRACT FROM AUTHOR]

Multi-class sentiment classification has extensive application backgrounds, whereas studies on this issue are still relatively scarce. In this paper, a framework for multi-class sentiment classification is proposed, which includes two parts: 1) selecting important features of texts using the feature selection algorithm, and 2) training multi-class sentiment classifier using the machine learning algorithm. Then, experiments are conducted for comparing the performances of four popular feature selection algorithms (document frequency, CHI statistics, information gain and gain ratio) and five popular machine learning algorithms (decision tree, naïve Bayes, support vector machine, radial basis function neural network and K-nearest neighbor) in multi-class sentiment classification. The experiments are conducted on three public datasets which include twelve data subsets, and 10-fold cross validation is used to obtain the classification accuracy concerning each combination of feature selection algorithm, machine learning algorithm, feature set size and data subset. Based on the obtained 3600 classification accuracies (4 feature selection algorithms × 5 machine learning algorithms × 15 feature set sizes × 12 data subsets), the average classification accuracy of each algorithm is calculated, and the Wilcoxon test is used to verify the existence of significant difference between different algorithms in multi-class sentiment classification. The results show that, in terms of classification accuracy, gain ratio performs best among the four feature selection algorithms and support vector machine performs best among the five machine learning algorithms. In terms of execution time, the similar comparisons are also conducted. The obtained results would be valuable for further improving the existing multi-class sentiment classifiers and developing new multi-class sentiment classifiers. [ABSTRACT FROM AUTHOR]

• A causal discovery algorithm based on rank correlation. • Apply Kendall correlation coefficient to causal discovery. • Lung nodule detection algorithm based on rank correlation. • Results demonstrate the feasibility of the proposed method. • The method can be applied to other medical tasks. Early diagnosis can significantly improve the survival rate of lung cancer patients. This study attempts to construct a causal structure network between the computational and semantic features of lung nodules through causal discovery algorithms, and to detect and prevent lung nodules based on this network. For complex and diverse lung nodule data sets, this paper proposes a new causal lung nodule detection algorithm Tau-CSFS in combination with rank correlation methods. The algorithm can effectively mine the causal relationship among lung cancer data that obey the non-linear non-Gaussian distribution, and the mixture of continuous and discrete variables, and has good predictive performance. We made three main contributions. First, we proved that the Kendall rank correlation coefficient that does not require data distribution can be used as a standard for independence test. Second, we applied Kendall rank correlation to Bayesian structure learning, and proposed a new causal discovery algorithm: Tau-CS algorithm based on hypothesis testing. The third contribution is to combine the Tau-CS algorithm with the feature selection method, and further propose the Tau-CSFS algorithm, which solves the problem of causality mining and diagnosis detection of lung nodule data. In the experiment, the Tau-CS algorithm is compared with the prior art on 7 Bayesian networks on the additive noise structure model, and it is proved that the algorithm has a better accuracy of causal structure learning. Finally, in the lung nodule detection stage, using the processed LIDC data set to perform two-classification and multi-classification experiments on seven semantic categories, the average accuracy of the Tau-CSFS algorithm reached 85.84% and 83.32%. The Tau-CSFS algorithm are better than comparable similar algorithms in the comprehensive performance index. The results show that the proposed algorithm has good detection performance and wide application prospects. [ABSTRACT FROM AUTHOR]

Optimization algorithms have shown significant advantages in solving diverse several real-world problems, especially where are limitations in computations and hardware requirements. However, they might get trapped in local optima or fast convergence. To solve these challenges, the exploration ability of an optimization algorithm should be evaluated and improved. This paper proposes a new approach for solving global optimization and feature selection problems to overcome these drawbacks and balance exploitation and exploration abilities. We propose a hybrid approach combining both, Gradient-Based Optimizer (GBO) using Slime Mould Algorithm (SMA), which exploits the strength factors of Gradient-Based Optimizer and Slime Mould Algorithm. The SMA is applied as a local search for the GBO to enhance its ability to explore more regions in the search space. This strategy is employed to achieve a balance between both exploration and exploitation. Two main experiments are applied to evaluate the performance of the GBOSMA. The first one solves a well-known global optimization problem (CEC2017), whereas the second one solves feature selection problems using several benchmark datasets. The GBOSMA is compared to the standard GBO, SMA, and the most recent optimization algorithms in both experiments. The results also show that the GBOSMA approach obtains promising results in both experiments outperforming other algorithms. Comparing with eight well-known optimization algorithms, the results (for both global optimization and feature selection problems) demonstrate that the GBOSMA overcomes others in terms of performance, speed and stability. • Improving the search ability of the GBO optimizer using the SMA algorithm. • Evaluating the performance of the proposed method using CEC2017 functions. • Selecting the most relative features to increase the classification accuracy. [ABSTRACT FROM AUTHOR]

With the rapid growth of the number of connected devices that exchange personal, sensitive, and important data through the IoT based global network, attacks that are targeting security services are increasing as well. Accordingly, there is a need for security solutions that are suitable for IoT environment. A network intrusion detection system (NIDS) is a solution that examines network traffic and alerts system administrators if there are security breaches. In this paper, an enhanced version of pigeon-inspired optimization (PIO) is proposed which enhances PIO by adding a local search algorithm named (LS-PIO). Moreover, an ensemble learning approach, which is based on multiple one-class classifiers, has been used in order to improve the performance of proposed NIDS. Four benchmark datasets were used to evaluate the LS-PIO and ensemble based NIDS which are BoT-IoT, UNSW-NB15, NLS-KDD and K D D C U P P 99 . The evaluation took into consideration F-score, accuracy, AUC, FPR and TPR. Results show that the suggested approach outperforms other NIDS techniques that are selected from state-of-the-art relevant research found in the literature. • Proposing a feature selection algorithm based on local search and pigeon optimizer. • Introducing an ensemble technique to used in IDS for IoT cyber-Attack. • Investigating Tabu and Hill Climbing search for improving PIO optimizer performance. • Anomaly detection based on one class classifiers has been explored. [ABSTRACT FROM AUTHOR]

In Data Mining, during the preprocessing step, there is a considerable diversity of candidate algorithms to select important features, according to some criteria. This broad availability of algorithms that perform the Feature Selection task gives rise to the difficulty of choosing, a priori , between the algorithms at hand, the most promising one for a particular problem. In this paper, we present the proposal and evaluation of a new architecture for the recommendation of Feature Selection algorithms based on the use of Metalearning. Our framework is very flexible since the user can adapt it to its proper needs. This flexibility is one of the main advantages of our proposal over other approaches in the literature, which involve steps that cannot be adapted to the user’s local requirements. Furthermore, it combines several concepts of intelligent systems, including Machine Learning and Data Mining, with topics derived from expert systems, as user and data-driven knowledge, with meta-knowledge. This set of solutions coupled with leading-edge technologies allows our architecture to be integrated into any information system, which impact on the automation of services and in reducing human effort during the process. Regarding the Metalearning process, our framework considers several types of properties inherent to the data sets, as well as, Feature Selection algorithms based on many information, distance, dependence and consistency measures. The quality of the methods for Feature Selection was estimated according to a multicriteria performance measure, which guided the ranking process of these algorithms for the construction of data metabases. Proposed by the authors of this work, this multicriteria performance measure combines any three measurements on a single one, creating an interesting and powerful tool to evaluate not only FS algorithms but also to assess any context where it is necessary a combination to maximize a measure or minimize it. The recommendation models, represented by decision trees and induced from the training metabases, allowed us to see in what circumstances a Feature Selection algorithm outperforms the other and what aspects of the data present greater influence in determining the performance of these algorithms. Nevertheless, if the user wishes, any other learning algorithm may be used to induce the recommendation model. This versatility is another strong point of this proposal. Results show that with the characterization of data, through statistical, information and complexity measures, it is possible to reach an accuracy higher than 90%. Besides yielding recommendation models that are interpretable and robust to overfitting, the developed architecture is less computationally expensive than approaches recently proposed in the literature. [ABSTRACT FROM AUTHOR]

With the advent of Big Data, data is being collected at an unprecedented fast pace, and it needs to be processed in a short time. To deal with data streams that flow continuously, classical batch learning algorithms cannot be applied and it is necessary to employ online approaches. Online learning consists of continuously revising and refining a model by incorporating new data as they arrive, and it allows important problems such as concept drift or management of extremely high-dimensional datasets to be solved. In this paper, we present a unified pipeline for online learning which covers online discretization, feature selection and classification. Three classical methods—the k -means discretizer, the χ 2 filter and a one-layer artificial neural network—have been reimplemented to be able to tackle online data, showing promising results on both synthetic and real datasets. [ABSTRACT FROM AUTHOR]

• Proposes Tri-Staged Feature Selection (TFS) for multi-class heterogeneous datasets. • Initial features are selected using Kruskal Wallis Test. • Refinement of obtained features using Memetic Algorithm with local beam search. • Final feature set refinement using Cuckoo search algorithm for better classification. • Experiments conducted on 12 real datasets for validation of proposed method. Classification algorithms and their preprocessing operations usually performs on feature selection on homogeneous or heterogeneous attributes, binary or multi-class labels separately. Only very few methods attempt to perform feature selection on datasets with heterogeneous multi-class attributes. In order to bridge this gap with better classification performance, the paper proposes a Tri-staged Feature Selection (TFS) methodology which performs (i) Feature selection using Kruskal Wallis test (ii) Refinement of feature selection using a new Memetic Algorithm with local beam search and genetic algorithm operations and (iii) Further refinement of feature selection using Cuckoo Search algorithm. Proper tradeoff between both exploration and exploitation is maintained in the proposed method. The experimental results on 12 datasets show that the proposed method is better than that of state-of-the-art methods used for feature selection in terms of multi-class accuracy, hamming loss, ranking loss, normalized coverage and convergence rate for multi-class heterogeneous datasets. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, application of the rough set theory (RST) to feature selection in customer relationship management (CRM) is introduced. Compared to other methods, the RST approach has the advantage of combining both qualitative and quantitative information in the decision analysis, which is extremely important for CRM. Automated decision support for CRM has been proposed in recent years. However, little work has been devoted to the development of computer-based systems to support CRM in rule induction. This paper presents a novel rough set based algorithm for automated decision support for CRM. Particularly, the approach is capable to handle real numbers instead of integer numbers through introduction of converted numbers involving tolerances. Being unique and useful in solving CRM problems, an alternative rule extraction algorithm (AREA) is presented for discovering preference-based rules according to the reducts which contain the maximum of strength index (SI) in the same case, where the data with tolerance. The empirical data set associated with CRM has proven the validity and reliability of these approaches. This research thus contributes to developing and validating a useful approach to automated decision support for CRM. This paper forms the basis for solving many other similar problems that occur in the service industry. [ABSTRACT FROM AUTHOR]

Feature selection techniques have been presented to allow us to choose a small subset of the original components' relevant features by removing irrelevant or redundant features. Feature selection is essential for many reasons such as simplification, performance, computational efficiency, and quality interpretability. Owing to the importance mentioned above, many researchers have proposed and developed many algorithms to solve the feature selection problem. Although these approaches produce useful results, they possess some shortcomings like inadequate feature reduction. In this paper, a novel feature selection algorithm based on the crow search algorithm is presented. The algorithm uses dynamic awareness probability to keep the balance between the local and global search processes. Moreover, a novel neighborhood assigning strategy has been introduced to optimize the local search. Considering the best-selected features in each iteration helps attain more benefits in global search. The main superiority of the proposed algorithm is the significant feature reduction along with retaining the accuracy. Compared to enhanced crow search algorithm, the proposed algorithm has improved the feature reduction metric and fitness metric by 27.12% and 5.16%, respectively, while losing the accuracy metric by only 0.53%. Several popular UCI datasets have been employed to evaluate the proposed feature selection algorithm. The experimental results show that the proposed algorithm outperformed other feature selection algorithms in state-of-the-art related works regarding feature reduction and accuracy. • Improve the balance between the local and global search. • Introducing a new neighborhood concept for improving the local search. • Proposing a new method to search more purposeful during exploration. • Increasing convergence rate using chaos. • Being pioneer in reducing the dataset volume. [ABSTRACT FROM AUTHOR]

• 3 layer neural networks outperform other ML algorithms in automated assessments. • In subjective assignments, deep neural networks can improve score prediction. • Word embeddings and linguistic feature results in more precise text mining models. • Extraction and selection of appropriate features can improve results. Machine learning (ML) has recently gained popularity in a variety of domains for automating tasks. This has also been used in the evaluation of student responses/ assignments. This paper presents a qualitatively enhanced methodology for automated score prediction of subjective assignments. To consider all the major aspects of a human grader, 21 linguistic features related to syntactical, grammatical, sentimental, and readability are analyzed from the assignments. This study makes use of the ASAP competition dataset from Kaggle. The evaluation metric used is Quadratic Weighted Kappa (QWK), which measures the agreement between the human graded score and the predicted score. The effect of appropriate feature selection has been observed using Mutual Information Regression. Four ML algorithms are investigated on identified linguistic features. 3 Layer Neural Network with feature selection performed well among all chosen ML algorithms with average QWK of 0.678. To include the benefits of deep learning, a new hybrid model (LF-BiLSTM-att-FS) is proposed that combines a higher level deep neural network (DNN) with the selected features. Pre-trained Glove embedding is used to include contextual information of the text. The proposed model results demonstrated an improvement in overall accuracy, with an average QWK value of 0.768. [ABSTRACT FROM AUTHOR]

• Feature selection algorithms based SOS algorithm are presented. • Four different algorithms are improved in order to select of optimal features. • MOSOS algorithm is the best to maximize accuracy while minimizing feature number. • Performance of the improved algorithms is evaluated on two well-known datasets. Feature selection is crucial to develop a brain computer interface (BCI) system which has high classification accuracy and less computational complexity in especially a large feature space. Feature selection (FS) problem has been solved by many various methods. Among these methods, especially evolutionary computation (EC) techniques have gained a lot of attention in recent years. However, there are very few studies in the literature that consider FS problem as a multi-objective problem to find the optimal trade-off between classification accuracy and the number of selected features. Therefore, in this paper, a non-dominated sorting multi-objective symbiotic organism search (NSMOSOS) algorithm is proposed to generate the optimal feature subset in BCI. The efficiency and robustness of the proposed algorithm as a feature selection method is investigated in two datasets based on motor imagery. The highest classification accuracies of NSMOSOS for dataset 1 and dataset 2 are obtained 97.86% with 11 features and 96.57% with average 19 features, respectively. The obtained results demonstrate that the proposed method achieves satisfying results with regard to both the classification accuracy improvement and feature reduction rates for both datasets. The superiority of the proposed method is verified compared with the existing methods for both datasets. Besides, three different versions of symbiotic search organism (SOS) algorithm are improved, and pros and cons of these algorithms are evaluated compared with each other. In conclusion, the paper indicates that the proposed NSMOSOS algorithm is an efficient and practicable technique for FS problem and could be helpful in developing the BCI applications. [ABSTRACT FROM AUTHOR]

• GBO-kNN is a proposed framework based on a wrapper approach for features selection. • The aim of the research is to reach maximum classification accuracy. • The performance of GBO-kNN is evaluated against real benchmark datasets. • The GBO-kNN performance is compared against seven recent metaheuristic algorithms. • Results showed high effectiveness on one dataset and moderate on another. Virtual Screening (VS) in drug discovery campaigns deals with enormously large databases containing massive redundant and/or irrelevant features. Thus, the pre-selection process of virtual screening requires developing a fast and precise classification methodology for filtering such a huge database. This paper proposed a framework depending on a wrapper selection approach for features selection. It consists of 1) an optimizer: Gradient-Based Optimizer (GBO), that hybridized with 2) the classifier: k-nearest neighbor (k-NN). The performance of the introduced framework, GBO-kNN, is estimated using real-world benchmark datasets; the QSAR Biodegradation which is composed of 41 features, and the Monoamine Oxidase (MAO) that consisted of 1665 features. the GBO-kNN framework is compared against seven recent swarm intelligence algorithms: Hybrid Harris Hawks Optimization Algorithm (HHO), Grey Wolf Optimization Algorithm (GWO), Butterfly Optimization Algorithm (BOA), Dragonfly Algorithm (DA), Moth-Flame Optimization Algorithm (MFO), Sine Cosine Algorithm (SCA), and Salp Swarm Algorithm (SSA) for reaching maximum classification accuracy. The experimental results exhibited that the proposed process, in comparison with other algorithms, achieved high effective accuracy of 98.8% over the high dimensional dataset (MAO) and a moderate effective performance over the low dimensional dataset (QSAR Biodegradation). The proposed framework source code is available at https://codeocean.com/capsule/9906421/tree/v1. [ABSTRACT FROM AUTHOR]

In the past two decades, the dimensionality of datasets involved in machine learning and data mining applications has increased explosively. Therefore, feature selection has become a necessary step to make the analysis more manageable and to extract useful knowledge about a given domain. A large variety of feature selection techniques are available in literature, and their comparative analysis is a very difficult task. So far, few studies have investigated, from a theoretical and/or experimental point of view, the degree of similarity/dissimilarity among the available techniques, namely the extent to which they tend to produce similar results within specific application contexts. This kind of similarity analysis is of crucial importance when two or more methods are combined in an ensemble fashion: indeed the ensemble paradigm is beneficial only if the involved methods are capable of giving different and complementary representations of the considered domain. This paper gives a contribution in this direction by proposing an empirical approach to evaluate the degree of consistency among the outputs of different selection algorithms in the context of high dimensional classification tasks. Leveraging on a proper similarity index, we systematically compared the feature subsets selected by eight popular selection methods, representatives of different selection approaches, and derived a similarity trend for feature subsets of increasing size. Through an extensive experimentation involving sixteen datasets from three challenging domains (Internet advertisements, text categorization and micro-array data classification), we obtained useful insight into the pattern of agreement of the considered methods. In particular, our results revealed how multivariate selection approaches systematically produce feature subsets that overlap to a small extent with those selected by the other methods. [ABSTRACT FROM AUTHOR]

Feature selection (FS) is recognized as one of the majority public and challenging problems in the Machine Learning domain. FS can be examined as an optimization problem that needs an effective optimizer to determine its optimal subset of more informative features. This paper proposes a wrapper FS method that combines chaotic maps (CMs) and binary Group Search Optimizer (GSO) called CGSO, which is used to solve the FS problem. In this method, five chaotic maps are incorporated with the GSO algorithm's main procedures, namely, Logistic, Piecewise, Singer, Sinusoidal, and Tent. The GSO algorithm is used as a search strategy, while k-NN is employed as an induction algorithm. The objective function is to integrate three main objectives: maximizing the classification accuracy value, minimizing the number of selected features, and minimizing the complexity of generated k-NN models. To evaluate the proposed methods' performance, twenty well-known UCI datasets are used and compared with other well-known published methods in the literature. The obtained results reveal the superiority of the proposed methods in outperforming other well-known methods, especially when using binary GSO with Tent CM. Finally, it is a beneficial method to be utilized in systems that require FS pre-processing. • A novel feature selection method is proposed for classification task. • Five Chaotic Group Search Optimizers are investigated. • Twenty UCI datasets were used in the experiments. • Our methods got a higher accuracy results using a fewer number of features. [ABSTRACT FROM AUTHOR]

A context-aware recommender system (CARS) utilizes users' context to provide personalized services. Contextual information can be derived from sensors in order to improve the accuracy of the recommendations. In this work, we focus on CARSs with high-dimensional contextual information that typically impacts the recommendation model, for example, by increasing the model's dimensionality and sparsity. Generating accurate recommendations is not enough to constitute a useful system from the user's perspective, since the use of some contextual information may cause problems, such as draining the user's battery, raising privacy concerns, and more. Previous studies suggested reducing the amount of contextual information utilized by using domain knowledge to select the most suitable information. This approach is only applicable when the set of contexts is small enough to handle and sufficient for preventing sparsity. Moreover, hand-crafted context information may not represent an optimal set of features for the recommendation process. Another approach is to compress the contextual information into a denser latent space, but this may limit the ability to explain the recommended items to the users or compromise their trust. In this paper, we present a multi-step approach for selecting low-dimensional subsets of contextual information and incorporating them explicitly within CARSs. At the core of our approach is a novel feature selection algorithm based on genetic algorithms, which outperforms state-of-the-art dimensionality reduction CARS algorithms by improving recommendation accuracy and interpretability. Over the course of evolution, thousands of diverse feature subsets are generated; a deep context-aware model is produced for each feature subset, and the subsets are stacked together. The resulting stacked model is accurate and only uses interpretable, explicit features. Our approach includes a mechanism of tuning the different underlying algorithms that affect user concerns, such as privacy and battery consumption. We evaluated our approach on two high-dimensional context-aware datasets derived from smartphones. An empirical analysis of our results confirms that our proposed approach outperforms state-of-the-art CARS models while improving transparency and interpretability for the user. In addition to the empirical results, we present several use cases, examples and methodology of how researchers, domain experts and CARS modelers can tweak the feature selection algorithm to improve various user concerns and interpretability. • A novel genetic algorithm-based approach for context-aware recommender systems. • A means of addressing user concerns, such as battery usage, privacy, etc. • Accurate and interpretable models that take users' concerns into account. • State-of-the-art qualitative and quantitative results on high-dimensional datasets. [ABSTRACT FROM AUTHOR]

Link prediction refers to predicting the connection probability between two nodes in terms of existing observable network information, such as network structural topology and node properties. Although traditional similarity-based methods are simple and efficient, their generalization performance varies widely in different networks. In this paper, we propose a novel link prediction approach ICP based on inductive matrix completion, which recoveries node connection probability matrix by applying node features to a low-rank matrix. The approach first explores a comprehensive node feature representation by combining different structural topology information with node importance properties via feature construction and selection. The selected node features are then used as the input of a supervised learning task for solving the low-rank matrix. The node connection probability matrix is finally recovered by a bi-linear function, which predicts the connection probability between two nodes with their features and the low-rank matrix. In order to demonstrate the ICP superiority, we took eleven related efforts including two recent methods proposed in 2020 as baseline methods, and it is shown that ICP has stable performance and good universality in twelve different real networks. Compared with the baseline methods, the improvements of ICP in terms of the average AUC results are ranging from 3.81% ∼ 12.77% and its AUC performance is improved by 0.08% ∼ 3.54% compared with the best baseline method. The limitation of ICP lies in its high computational complexity due to the feature construction, but the complexity can be reduced by replacing complex features with node semantic attributes if there are additional data available. Moreover, it provides a potential link prediction solution for large-scale networks, since inductive matrix completion is a supervised learning task, in which the underlying low-rank matrix can be solved by representative nodes instead of all their nodes. • A novel link prediction approach based on inductive matrix completion is proposed. • A comprehensive node feature representation is employed. • Link prediction is based on the node features and the factorized matrices. • Comparisons have been done with both classical and state-of-the-art methods. • The approach provides a potential link prediction solution for large-scale networks. [ABSTRACT FROM AUTHOR]

• A new binary social spider algorithm (BinSSA) has been developed. • BinSSA is developed by adding a crossover operator. • The developed algorithm has shown superior performance. The social spider algorithm (SSA) is a heuristic algorithm created on spider behaviors to solve continuous problems. In this paper, firstly a binary version of the social spider algorithm called binary social spider algorithm (BinSSA) is proposed. Currently, there is insufficient focus on the binary version of SSA in the literature. The main part of the binary version is the transfer function. The transfer function is responsible for mapping continuous search space to binary search space. In this study, eight of the transfer functions divided into two families, S-shaped and V-shaped, are evaluated. BinSSA is obtained from SSA, by transforming constant search space to binary search space with eight different transfer functions (S-Shapes and V-Shaped). Thus, eight different variations of BinSSA are formed as BinSSA1, BinSSA2, BinSSA3, BinSSA4, BinSSA5, BinSSA6, BinSSA7, and BinSSA8. For increasing, exploration and exploitation capacity of BinSSA, a crossover operator is added as BinSSA-CR. In secondly, the performances of BinSSA variations are tested on feature selection task. The optimal subset of features is a challenging problem in the process of feature selection. In this paper, according to different comparison criteria (mean of fitness values, the standard deviation of fitness values, the best of fitness values, the worst of fitness values, accuracy values, the mean number of the selected features, CPU time), the best BinSSA variation is detected. In the feature selection problem, the K-nearest neighbor (K-NN) and support vector machines (SVM) are used as classifiers. A detailed study is performed for the fixed parameter values used in the fitness function. BinSSA is evaluated on low-scaled, middle-scaled and large-scaled twenty-one well-known UCI datasets and obtained results are compared with state-of-art algorithms in the literature. Obtained results have shown that BinSSA and BinSSA-CR show superior performance and offer quality and stable solutions. [ABSTRACT FROM AUTHOR]

Silent diseases is an umbrella term that captures a spectrum of chronic illnesses that produce no clinically obvious signs and are diagnosed at advanced stages when the damage is irreversible. Current diagnostic strategies of silent diseases depend on self-reported symptoms and observed behavior through extended periods of time, and until now there are no specific clinical tests to diagnose silent diseases. Scientific research suggests the importance of early diagnosis to restore the functionality and reduce diseases-related complications. Previous studies primarily focused on feature selection methods to aid in medical diagnosis. Traditional feature selection methods are primarily focused on correct classification and often ignore features' costs; the cost of clinical tests required to acquire the feature value. However, in medical diagnosis, features have different associated costs. Because ignoring features' costs may result in a high cost diagnostic strategy that cannot be used in practice, developing a low-cost diagnostic strategy remains a subject of much interest. In this paper, new Mixed Integer Programming (MIP) models, namely, Cost-sensitive Support Vector Machine (CS-SVM) and Cost-sensitive Multi-surface Method Tree (CS-MSMT) that allow for simultaneous selection of low-cost and informative features are proposed. The CS-SVM and CS-MSMT are superior because they have the ability to account for shared costs. The CS-SVM and CS-MSMT were modified to embed shared costs across feature groups, and are termed Discounted CS-SVM (d CS-SVM) and Discounted CS-MSMT (d CS-MSMT), respectively. Computationally effective algorithm that integrates aggressive bound tightening with the MIP formulation is proposed. To demonstrate the effectiveness of the proposed models, different analysis paradigms are conducted on six UCI medical datasets; Chronic Kidney Disease, Hepatitis, Heart Disease, Thyroid, Diabetes and Leukemia. The results demonstrate the efficiency and robustness of the CS-SVM and CS-MSMT (and consequently the d CS-SVM and d CS-MSMT) under various conditions. The CS-SVM and CS-MSMT improved accuracy by 10.3% and 3.4% and reduced costs by 94.3% and 72.4% in the leukemia dataset, respectively. • New MIP models for cost-sensitive feature selection are proposed. • The models are robust enough to account for shared cost across feature groups. • Aggressive bound tightening within Branch and Cut algorithm was used. • The proposed cost-sensitive feature selection models outperformed existing feature selection techniques. • The models improved the accuracy up by 10.3% and decreased the cost up to 96%. [ABSTRACT FROM AUTHOR]

• Enhance EO algorithm using ReliefF algorithm and the local search strategy. • Propose new feature selection method based on a hybridization RBEO-LS method. • Evaluate RBEO-LS using 16 UCI datasets and 10 high dimensional biological datasets. • Results show superiority of RBEO-LS among other state-of-the-art methods. Feature selection (FS) is an important task in any classification process and aims to choose the smallest features number that yields higher classification accuracy. FS can be formulated as a combinatorial NP-hard problem for which robust metaheuristics are used as efficient wrapper-based FS approaches. However, when applied for high dimensional datasets that present large features number and few samples, the effectiveness of such wrapper-metaheuristics degraded, and their computation costs increased. To tackle this problem, we propose in this paper a hybrid FS approach based on the ReliefF filter method and a novel metaheuristic Equilibrium Optimizer (EO). The proposed method, called RBEO-LS, is composed of two phases. In the first phase, the ReliefF algorithm is used as a preprocessing step to assign weights for features, which estimate their relevance to the classification task. In the second phase, the binary EO (BEO) is used as a wrapper search approach. The features are ranked according to their weights and are used for the initialization of the BEO population. We embedded the BEO with a local search strategy to improve its performance by adding relevant features and removing redundant ones from the features subset guided by the features ranking and the Pearson coefficient correlation. The performance of the developed algorithm has been evaluated on sixteen UCI datasets and ten high dimensional biological datasets. The UCI datasets contain a high number of samples and a small or medium number of features. The biological datasets present a high number of features with few samples. The results demonstrate that the use of the ReliefF algorithm and the local search strategy improves the performance of the EO algorithm. The results also show the superiority of the RBEO-LS, among other state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

• FCS is proposed as a complementary step for principal component analysis. • PCA-FCS identifies the most relevant components based on their level of contribution. • PCA-FCS simultaneously performs dimension reduction and component selection. • PCA-FCS has substantially improved the accuracy of the random forest algorithm. Principal component analysis (PCA) is one the most effective and widely used dimensionality-reduction techniques which aggregates the maximal variance in the first few components. In many applications, the first few components of PCA are used in place of the original variables in statistical and machine learning models; nevertheless, such use does not guarantee the selection of the most relevant components to the target variable. This paper presents an efficient approach in which all components, rather than simply the first few, are considered as input for the random forest (RF) model, and a feature selection algorithm is integrated with the RF to select the most relevant components, hence called fully component selection (FCS). The proposed method was evaluated on spectroscopic data comprising 70 soil samples with 2050 spectral bands to estimate soil organic carbon (SOC). The FCS approach was compared to the RF model once considering the original features, then again using only the first few components, and again using all components without feature selection. The results showed that RF-FCS substantially outperformed other approaches, such that the R2 was increased between 25.2% and 55.5%. Furthermore, the findings indicated that the most relevant principal components (PCs) to the target variable were not the first few, but PC6, PC7, PC8, PC15, and PC43. Using the FCS approach increased model performance substantively, and coupling it with machine learning and statistical models is strongly recommended for high dimensional data applications. [ABSTRACT FROM AUTHOR]

Feature selection in real-world data mining problems is essential to make the learning task efficient and more accurate. Identifying the best feature selection algorithm, among the many available, is a complex activity that still relies heavily on human experts or some random trial-and-error procedure. Thus, the automated machine learning community has taken some steps towards the automation of this process. In this paper, we address the metalearning challenge of recommending feature selection algorithms by proposing a novel meta-feature engineering model. Our model considers a broad collection of meta-features that enable the study of the relationship between the dataset properties and the feature selection algorithm performance in terms of several criteria. We arrange the input meta-features into eight categories: (i) simple, (ii) statistical, (iii) information-theoretical, (iv) complexity, (v) landmarking, (vi) based on symbolic models, (vii) based on images, and (viii) based on complex networks (graphs). The target meta-features emerge from a multi-criteria performance measure, based on five individual performance indexes, that assesses feature selection methods grounded in information, distance, dependence, consistency, and precision measures. We evaluate our proposal using a recently developed framework that extracts the input meta-features from 213 benchmark datasets, and ranks the assessed feature selection algorithms, to fill in the target meta-features in meta-bases. This evaluation uses five state-of-the-art classification methods to induce recommendation models from meta-bases: C4.5, Random Forest, XGBoost, ANN, and SVM. The results showed that it is possible to reach an average accuracy of up to 90% applying our meta-feature engineering model. This work is the first to use an extensive empirical evaluation to provide a careful discussion of the strengths and limitations of more than 160 meta-features. These meta-features, while designed to aid the task of feature selection algorithm recommendation, can readily be employed in other metalearning scenarios. Therefore, we believe our findings are a valuable contribution to the fields of automated machine learning and data mining, as well as to the feature extraction and pattern recognition communities. • A novel meta-feature engineering model recommends feature selection algorithms. • The proposal obtains promising results from 213 datasets with hit rates of up to 90%. • Some simple, landmarking, image, and graph-based input meta-features highlighted. • A multi-criteria performance measure rigorously assesses candidate algorithms. • Chains of binary or multiclass classifiers can efficiently rank candidate algorithms. [ABSTRACT FROM AUTHOR]

• A novel multi-objective feature selection algorithm is proposed. • It hybridizes filter and wrapper models into whale optimization algorithm. • It has been validated through ten datasets and compared against five algorithms. • Yields solutions with fewer features and improves the classification accuracy. Feature selection aims at finding the minimum number of features that result in high classification accuracy. Accordingly, the feature selection is considered as a multi-objective problem. However, most existing approaches treat feature selection as single-objective problem, and they are often divided into two main categories: filter and wrapper methods. Filters are known as fast methods but less accurate, while wrappers are computationally expensive but with high classification performance. This paper proposes a novel hybrid filter-wrapper feature selection approach using whale optimization algorithm (WOA). The proposed method is a multi-objective algorithm in which a filter and wrapper fitness functions are optimized simultaneously. Our algorithm's efficiency is demonstrated through an extensive comparison with seven well-known algorithms on twelve benchmark datasets. Experimental results show the ability of the proposed algorithm to obtain several subsets that include smaller number of features with excellent classification accuracy. [ABSTRACT FROM AUTHOR]

• Efficient and technically sound feature selection based on Sobol decomposition. • The proposed method takes into account interactions between features. • Superior performance vis-a-vis other state of the art sampling techniques. • The proposed method is built on solid mathematical framework. Existing feature selection methods fail to properly account for interactions between features. In this paper, we attempt to remedy this issue by using orthogonal variance decomposition to evaluate features. The orthogonality of the decomposition allows us to directly calculate the total contribution of each feature to the output variance. As a result, we obtain an efficient and technically sound feature selection algorithm which takes into account feature interactions. The proposed algorithm has low computational complexity compared to other methods used in the literature. Numerical experiments demonstrate that our method accurately identifies relevant features and improves the accuracy of numerical models. [ABSTRACT FROM AUTHOR]

Abstract: Feature selection and classification techniques have been studied independently without considering the interaction between both procedures, which leads to a degraded performance. In this paper, we present a new neural network approach, which is called an algorithm learning based neural network (ALBNN), to improve classification accuracy by integrating feature selection and classification procedures. In general, a knowledge-based artificial neural network operates on prior knowledge from domain experience, which provides it with better starting points for the target function and leads to better classification accuracy. However, prior knowledge is usually difficult to identify. Instead of using unknown background resources, the proposed method utilizes prior knowledge that is mathematically calculated from the properties of other learning algorithms such as PCA, LARS, C4.5, and SVM. We employ the extreme learning machine in this study to help obtain better initial points faster and avoid irrelevant time-consuming work, such as determining architecture and manual tuning. ALBNN correctly approximates a target hypothesis by both considering the interaction between two procedures and minimizing individual procedure errors. The approach produces new relevant features and improves the classification accuracy. Experimental results exhibit improved performance in various classification problems. ALBNN can be applied to various fields requiring high classification accuracy. [Copyright &y& Elsevier]

Abstract: Feature selection is one of the core issues in designing pattern recognition and machine learning systems, and has attracted considerable attention in the literature. In this paper, a new feature subset selection algorithm with conditional mutual information is proposed, which firstly guarantees to find a subset of which the mutual information with the class is the same as that of the original set of features, and then eliminates potential redundant features from the view of minimal information loss based on the cumulate conditional mutual information minimization criterion. From the reliability point of view, this criterion can also abate the disturbance caused by sample insufficiency in conditional mutual information estimation. In addition, a fast implementation of conditional mutual information estimation is proposed and used to tackle the computationally intractable problem. Empirical results verify that our algorithm is efficient and achieves better accuracy than several representative feature selection algorithms for three typical classifiers on various datasets. [Copyright &y& Elsevier]

Abstract: Automatic communication signal recognition plays an important role for many novel computer and communication technologies. Most of the proposed techniques can only identify a few kinds of digital signal and/or low order of them. They usually require high levels of signal to noise ratio (SNR). In this paper, we investigate twofold. First, we propose an efficient system that uses a combination set of spectral characteristics and higher order moments up to eighth and higher order cumulants up to eighth as the effective features. As the classifier we used a multi-layer perceptron (MLP) neural network. In this stage we investigate different learning algorithms of MLP neural networks that some of them, such as quick prop (QP) learning algorithm, extended delta-bar-delta (EDBD), super self adaptive back propagation (SuperSAB) and conjugate gradient (CG) are proposed for the first time in the area of communication signals recognition. Experimental results show that proposed system discriminates a lot of digital communication signals with high accuracy even at very low SNRs. But a lot of features are used for this recognition. Then at the second fold, in order to reduce the complexity of the recognizer, we have proposed a novel hybrid intelligent technique. In this technique we have optimized the classifier design by Bees Algorithm (BA) for selection of the best features that are fed to the classifier. Simulation results show that the proposed technique has very high recognition accuracy with seven features selected by BA. [ABSTRACT FROM AUTHOR]

Abstract: The inclusion of irrelevant, redundant, and inconsistent features in the data-mining model results in poor predictions and high computational overhead. This paper proposes a novel information theoretic-based interact (IT-IN) algorithm, which concerns the relevance, redundancy, and consistency of the features. The proposed IT-IN algorithm is compared with existing Interact, FCBF, Relief and CFS feature selection algorithms. To evaluate the classification accuracy of IT-IN and remaining four feature selection algorithms, Naïve Bayes, SVM, and ELM classifier are used for ten UCI repository datasets. The proposed IT-IN performs better than existing above algorithms in terms of number of features. The specially designed hash function is used to speed up the IT-IN algorithms and provides minimum computation time than the Interact algorithms. The result clearly reveals that the proposed feature selection algorithm improves the classification accuracy for ELM, Naïve Bayes, and SVM classifiers. The performance of proposed IT-IN with ELM classifier is superior to other classifiers. [ABSTRACT FROM AUTHOR]

Abstract: Feature selection is a key issue in the research on rough set theory. However, when handling large-scale data, many current feature selection methods based on rough set theory are incapable. In this paper, two novel feature selection methods are put forward based on decomposition and composition principles. The idea of decomposition and composition is to break a complex table down into a master-table and several sub-tables that are simpler, more manageable and more solvable by using existing induction methods, then joining them together in order to solve the original table. Compared with some traditional methods, the efficiency of the proposed algorithms can be illustrated by experiments with standard datasets from UCI database. [ABSTRACT FROM AUTHOR]

Abstract: There are irrelevant features that are redundant or significantly degrade the learning accuracy in the real-world complex classification tasks. This paper presents a new hybrid learning algorithm based on a cooperative coevolutionary algorithm (Co-CEA) with dual populations for designing the radial basis function neural network (RBFNN) models with an explicit feature selection. This approach attempts to complete both the RBFNN construction and the feature selection simultaneously. The proposed algorithm utilizes the Co-CEA’s divide-and-cooperative mechanism, which utilizes the evolutionary algorithms executing in parallel to coevolve subpopulations, corresponding to the hidden layer structure and the dominate features respectively. The algorithm adopts the binary encoding to represent the feature subset and the matrix-form mixed encoding to represent the RBFNN hidden layer structure, and a complete solution is formed via collaborations among the two subpopulations. Experimental results illustrate that the proposed algorithm outperforms other algorithms in references in terms of the classification accuracy, and it is able to obtain both prominent features and good RBFNN structure with higher prediction capability. [Copyright &y& Elsevier]

Abstract: The problem of feature selection is to find a subset of features for optimal classification. A critical part of feature selection is to rank features according to their importance for classification. The Support Vector Machine (SVM) has been applied to a number of applications, such as bioinformatics, face recognition, text categorization, handwritten digit recognition, and so forth. Based on the success of the SVM, several feature selection algorithms that use it have recently been proposed. This paper proposes a new feature-ranking algorithm based on support vectors (SVs). Support vectors refer to those sample vectors that lie around the decision boundary between two different classes. Although SV-based feature ranking can be applied to any discriminant analysis, two linear discriminants are considered here: Fisher’s linear discriminant and the Support Vector Machine. Features are ranked based on the weight associated with each feature or as determined by recursive feature elimination. The experiments show that our feature-ranking algorithms are competitive in accuracy with the existing methods and much faster. [Copyright &y& Elsevier]

Abstract: Rough set algorithm was used as a new methodology to build structure–activity relationship (SAR) models in this paper. It acted as feature selector and nonlinear rule generator. The SAR model expressed as human readable if-then rules was developed for the inhibition of the serine/threonine kinase CDK1/cyclinB by compounds from the indirubin inhibitor family. The feature selection ability of rough set algorithm was compared with the build-in approaches (CfsSubsetEval and ConsistencySubsetEval) in Weka under leave-one-out (LOO) and 10-fold cross-validation. Through training a set of 31 objects, a rule-based SAR model had been built with a reduct generated by rough set. The predictability of the model was evaluated by an external test set of 16 compounds. The existing powerful approaches, such as the decision tree learners, neural network, support vector classifier and LogitBoost approaches, were used to verify the performance of rough set method. It revealed that rough set method should play important role in data preprocessing and model building of nonlinear SAR analysis. The advantages and limitations of rough set-based SAR analysis were discussed. The results were satisfactorily in accordance with the available understanding of cocrystal structures and 3D QSAR models. [Copyright &y& Elsevier]

Abstract: Algorithms for feature selection in predictive data mining for classification problems attempt to select those features that are relevant, and are not redundant for the classification task. A relevant feature is defined as one which is highly correlated with the target function. One problem with the definition of feature relevance is that there is no universally accepted definition of what it means for a feature to be ‘highly correlated with the target function or highly correlated with the other features’. A new feature selection algorithm which incorporates domain specific definitions of high, medium and low correlations is proposed in this paper. The proposed algorithm conducts a heuristic search for the most relevant features for the prediction task. [Copyright &y& Elsevier]

• We have designed a method for ensemble feature selection. • We model the feature selection process to a Pareto-based optimization problem. • The crowding distance between solutions is the secondary measure. • The method is an ensemble of relevancy and redundancy methods. • The proposed PEFS method outperforms competitive algorithms. In this paper, ensemble feature selection is modeled as a bi-objective optimization problem regarding features' relevancy and redundancy degree. The proposed method, which is called PEFS, first uses the modeled bi-objective optimization problem to find the non-dominated features based on the decision matrix constructed by different feature selection algorithms. In the second step, the found non-dominated features are sorted using the crowding distance in the bi-objective space. These sorted features remove from the feature space, and the process of finding the non-dominated features will continue until all the features are sorted. To illustrate the optimality and efficiency of the proposed method, we have compared our approach with some ensemble feature selection methods and basic algorithms used in the ensemble process. The results show that our method in terms of accuracy and F-score is superior to other similar methods and performs in a short running-time. [ABSTRACT FROM AUTHOR]