Search

In real-world applications, feature selection is crucial for enhancing the performance of data science and machine learning models. Typically, feature selection is a complex combinatorial optimization problem and a multi-objective optimization problem. Its primary goals are to reduce the dimensionality of the dataset and enhance the performance of machine learning algorithms. The selection of features in high-dimensional datasets is challenging due to the intricate relationships between features, which pose significant challenges to the performance and computational efficiency of algorithms. This paper introduces a Knowledge-Guided Competitive Co-Evolutionary Algorithm (KCCEA) for feature selection, especially for high-dimensional features. In the proposed algorithm, we make improvements to the foundational dominance-based multi-objective evolutionary algorithm in two aspects. First, the use of feature correlation as knowledge to guide evolution enhances the search speed and quality of traditional multi-objective evolutionary algorithm solutions. Second, a dynamically allocated competitive–cooperative evolutionary mechanism is proposed, integrating the improved knowledge-guided evolution with traditional evolutionary algorithms, further enhancing the search efficiency and diversity of solutions. Through rigorous empirical testing on various datasets, the KCCEA demonstrates superior performance compared to basic multi-objective evolutionary algorithms, providing effective solutions to multi-objective feature selection problems while enhancing the interpretability and effectiveness of prediction models. [ABSTRACT FROM AUTHOR]

When performing classification tasks on high-dimensional data, traditional machine learning algorithms often fail to filter out valid information in the features adequately, leading to low levels of classification accuracy. Therefore, this paper explores the high-dimensional data from both the data feature dimension and the model ensemble dimension. We propose a high-dimensional ensemble learning classification algorithm focusing on feature space reconstruction and classifier ensemble, called the HDELC algorithm. First, the algorithm considers feature space reconstruction and then generates a feature space reconstruction matrix. It effectively achieves feature selection and reconstruction for high-dimensional data. An optimal feature space is generated for the subsequent ensemble of the classifier, which enhances the representativeness of the feature space. Second, we recursively determine the number of classifiers and the number of feature subspaces in the ensemble model. Different classifiers in the ensemble system are assigned mutually exclusive non-intersecting feature subspaces for model training. The experimental results show that the HDELC algorithm has advantages compared with most high-dimensional datasets due to its more efficient feature space ensemble capability and relatively reliable ensemble operation performance. The HDELC algorithm makes it possible to solve the classification problem for high-dimensional data effectively and has vital research and application value. [ABSTRACT FROM AUTHOR]

In the era of big data, the demand for multivariate time series prediction has surged, drawing increased attention to feature selection and neural networks in machine learning. However, certain feature selection methods neglect the alignment between actual data sample differences and clustering results, while neural networks lack automatic parameter adjustment in response to changing target features. This paper presents the MBSSA-Bi-AESN model, a Bi-directional Adaptive Echo State Network that utilizes the modified salp swarm algorithm (MBSSA) and feature selection to address the limitations of manually set parameters. Initial feature subset selection involves assigning weights based on the consistency of clustering results with differences. Subsequently, the four critical parameters in the Bi-AESN model are optimized using MBSSA. The optimized Bi-AESN model and selected feature subset are then integrated for simultaneous model learning and optimal feature subset selection. Experimental analysis on eight datasets demonstrates the superior prediction accuracy of the MBSSA-Bi-AESN model compared to benchmark models, underscoring its feasibility, validity, and universality. [ABSTRACT FROM AUTHOR]

In Machine Learning (ML), Feature Selection (FS) plays a crucial part in reducing data's dimensionality and enhancing any proposed framework's performance. However, in real-world applications, FS work suffers from high dimensionality, computational and storage complexity, noisy or ambiguous nature, high performance, etc. The area of FS is very vast and challenging in its nature. There are lots of work that have been reported on FS over the various area of applications. This paper has discussed FS's framework and the multiple models of FS with detailed descriptions. We have also classified the various FS algorithms with respect to the data, i.e., structured or labeled data and unstructured data for the different applications of ML. We have also discussed what essential features are, the commonly used FS methods, the widely used datasets, and the widely used work done in the various ML fields for the FS task. Here we try to view the multiple comparison experimental results of FS work in different result discussions. This paper draws a descriptive survey on FS with the associated area of real-world problem domains. This paper's main objective is to understand the main idea of FS work and identify the core idea of how FS will be applicable in various problem domains. [ABSTRACT FROM AUTHOR]

The efficiency and the effectiveness of a machine learning (ML) model are greatly influenced by feature selection (FS), a crucial preprocessing step in machine learning that seeks out the ideal set of characteristics with the maximum accuracy possible. Due to their dominance over traditional optimization techniques, researchers are concentrating on a variety of metaheuristic (or evolutionary) algorithms and trying to suggest cutting-edge hybrid techniques to handle FS issues. The use of hybrid metaheuristic approaches for FS has thus been the subject of numerous research works. The purpose of this paper is to critically assess the existing hybrid FS approaches and to give a thorough literature review on the hybridization of different metaheuristic/evolutionary strategies that have been employed for supporting FS. This article reviews pertinent documents on hybrid frameworks that were published in the period from 2009 to 2022 and offers a thorough analysis of the used techniques, classifiers, datasets, applications, assessment metrics, and schemes of hybridization. Additionally, new open research issues and challenges are identified to pinpoint the areas that have to be further explored for additional study. [ABSTRACT FROM AUTHOR]

CC's (Cloud Computing) networks are distributed and dynamic as signals appear/disappear or lose significance. MLTs (Machine learning Techniques) train datasets which sometime are inadequate in terms of sample for inferring information. A dynamic strategy, DevMLOps (Development Machine Learning Operations) used in automatic selections and tunings of MLTs result in significant performance differences. But, the scheme has many disadvantages including continuity in training, more samples and training time in feature selections and increased classification execution times. RFEs (Recursive Feature Eliminations) are computationally very expensive in its operations as it traverses through each feature without considering correlations between them. This problem can be overcome by the use of Wrappers as they select better features by accounting for test and train datasets. The aim of this paper is to use DevQLMLOps for automated tuning and selections based on orchestrations and messaging between containers. The proposed AKFA (Adaptive Kernel Firefly Algorithm) is for selecting features for CNM (Cloud Network Monitoring) operations. AKFA methodology is demonstrated using CNSD (Cloud Network Security Dataset) with satisfactory results in the performance metrics like precision, recall, F-measure and accuracy used. [ABSTRACT FROM AUTHOR]

Keeping computer reliability to confirm reliable, secure, and truthful correspondence of data between different enterprises is a major security issue. Ensuring information correspondence over the web or computer grids is always under threat of hackers or intruders. Many techniques have been utilized in intrusion detections, but all have flaws. In this paper, a new hybrid technique is proposed, which combines the Ensemble of Feature Selection (EFS) algorithm and Teaching LearningBased Optimization (TLBO) techniques. In the proposed, EFSTLBO method, the EFS strategy is applied to rank the features for choosing the ideal best subset of applicable information, and the TLBO is utilized to identify the most important features from the produced datasets. The TLBO algorithm uses the Extreme Learning Machine (ELM) to choose the most effective attributes and to enhance classification accuracy. The performance of the recommended technique is evaluated in a benchmark dataset. The experimental outcomes depict that the proposed model has high predictive accuracy, detection rate, false-positive rate, and requires less significant attributes than other techniques known from the literature. [ABSTRACT FROM AUTHOR]

In this paper, some significant efforts on fuzzy oblique decision tree (FODT) have been done to improve classification accuracy and decrease tree size. Firstly, to eliminate data redundancy and improve classification efficiency, a forward greedy fast feature selection algorithm based on neighborhood rough set (NRS_FS_FAST) is introduced. Then, a new fuzzy rule generation algorithm (FRGA) is proposed to generate fuzzy rules. These fuzzy rules are used to construct leaf nodes for each class in each layer of the FODT. Different from the traditional axis-parallel decision trees and oblique decision trees, the FODT takes dynamic mining fuzzy rules as decision functions. Moreover, the parameter δ, which can control the size of the tree, is optimized by genetic algorithm. Finally, a series of comparative experiments are carried out with five traditional decision trees (C4.5, Best First Tree (BFT), amulti-class alternating decision tree (LAD), Simple Cart (SC), Naive Bayes Tree (NBT)), and recently proposed decision trees (FRDT, HHCART, and FMMDT-HB) on UCI machine learning datasets. The experimental results demonstrate that the FODT exhibits better performance on classification accuracy and tree size than the chosen benchmarks. [ABSTRACT FROM AUTHOR]

Feature selection becomes prominent, especially in the data sets with many variables and features. It will eliminate unimportant variables and improve the accuracy as well as the performance of classification. Random Forest has emerged as a quite useful algorithm that can handle the feature selection issue even with a higher number of variables. In this paper, we use three popular datasets with a higher number of variables (Bank Marketing, Car Evaluation Database, Human Activity Recognition Using Smartphones) to conduct the experiment. There are four main reasons why feature selection is essential. First, to simplify the model by reducing the number of parameters, next to decrease the training time, to reduce overfilling by enhancing generalization, and to avoid the curse of dimensionality. Besides, we evaluate and compare each accuracy and performance of the classification model, such as Random Forest (RF), Support Vector Machines (SVM), K-Nearest Neighbors (KNN), and Linear Discriminant Analysis (LDA). The highest accuracy of the model is the best classifier. Practically, this paper adopts Random Forest to select the important feature in classification. Our experiments clearly show the comparative study of the RF algorithm from different perspectives. Furthermore, we compare the result of the dataset with and without essential features selection by RF methods varImp(), Boruta, and Recursive Feature Elimination (RFE) to get the best percentage accuracy and kappa. Experimental results demonstrate that Random Forest achieves a better performance in all experiment groups. [ABSTRACT FROM AUTHOR]

Adaptive boosting (Adaboost) is a typical ensemble learning algorithm, which has been studied and widely used in classification tasks. Traditional Adaboost algorithms ignore the sample weights while selecting the most useful features, and most of them ignore the fact that the performances of weak classifiers on each category are always different. On this basis, a weighted feature selection and category classification confidence based Adaboost algorithm is proposed in this paper. The first contribution, is that we propose a weighted feature selection to select the most useful features, which can both distinguish the majority of all samples and the previous misclassified samples. The second contribution, is that we improve the traditional error rate calculation method and propose a category based error rate calculation method to combine the classification abilities of Adaboost on different categories. A detailed performances comparison of various Adaboost algorithms are carried out on eight typical datasets. The experimental results show that the proposed algorithm obtains significant improvement on classification accuracy compared to typical Adaboost algorithms when different datasets especially the unbalanced datasets are used. [ABSTRACT FROM AUTHOR]

Feature selection is one of the main steps in preprocessing data in machine learning, and its goal is to reduce features by removing additional and noisy features. Feature selection methods and feature reduction in a dataset must consider the accuracy of the classifying algorithms. Meta-heuristic algorithms serve as the most successful and promising methods to solve this problem. Symbiotic Organisms Search (SOS) is one of the most successful meta-heuristic algorithms inspired by organisms' interaction in nature called mutualism, commensalism, and parasitism. In this paper, three SOS-based binary approaches are offered to solve the feature selection problem. In the first and second approaches, several S-shaped transfer functions and several Chaotic Tent Function-based V-shaped transfer functions called BSOSST and BSOSVT are used to make the binary SOS (BSOS). In the third approach, an advanced BSOS based on changing SOS and the chaotic Tent function operators called EBCSOS is provided. The EBCSOS algorithm uses the chaotic Tent function and the Gaussian mutation to increase usefulness and exploration. Moreover, two new operators, i.e., BMPT and BCPT, are suggested to make the commensalism and mutualism stage binary based on a chaotic function to solve the feature selection problem. Finally, the proposed BSOSST and BSOSVT methods and the advanced version of EBCSOS were implemented on 25 datasets than the basic algorithm's binary meta-heuristic algorithms. Various experiments demonstrated that the proposed EBCSOS algorithm outperformed other methods in terms of several features and accuracy. To further confirm the proposed EBCSOS algorithm, the problem of detecting spam E-mails was applied, with the results of this experiment indicating that the proposed EBCSOS algorithm significantly improved the accuracy and speed of all categories in detecting spam E-mails. [ABSTRACT FROM AUTHOR]

There is a consistent rise in chronic diseases worldwide. These diseases decrease immunity and the quality of daily life. The treatment of these disorders is a challenging task for medical professionals. Dimensionality reduction techniques make it possible to handle big data samples, providing decision support in relation to chronic diseases. These datasets contain a series of symptoms that are used in disease prediction. The presence of redundant and irrelevant symptoms in the datasets should be identified and removed using feature selection techniques to improve classification accuracy. Therefore, the main contribution of this paper is a comparative analysis of the impact of wrapper and filter selection methods on classification performance. The filter methods that have been considered include the Correlation Feature Selection (CFS) method, the Information Gain (IG) method and the Chi-Square (CS) method. The wrapper methods that have been considered include the Best First Search (BFS) method, the Linear Forward Selection (LFS) method and the Greedy Step Wise Search (GSS) method. A Decision Tree algorithm has been used as a classifier for this analysis and is implemented through the WEKA tool. An attribute significance analysis has been performed on the diabetes, breast cancer and heart disease datasets used in the study. It was observed that the CFS method outperformed other filter methods concerning the accuracy rate and execution time. The accuracy rate using the CFS method on the datasets for heart disease, diabetes, breast cancer was 93.8%, 89.5% and 96.8% respectively. Moreover, latency delays of 1.08 s, 1.02 s and 1.01 s were noted using the same method for the respective datasets. Among wrapper methods, BFS' performance was impressive in comparison to other methods. Maximum accuracy of 94.7%, 95.8% and 96.8% were achieved on the datasets for heart disease, diabetes and breast cancer respectively. Latency delays of 1.42 s, 1.44 s and 132 s were recorded using the same method for the respective datasets. On the basis of the obtained result, a new hybrid Attribute Evaluator method has been proposed which effectively integrates enhanced K-Means clustering with the CFS filter method and the BFS wrapper method. Furthermore, the hybrid method was evaluated with an improved decision tree classifier. The improved decision tree classifier combined clustering with classification. It was validated on 14 different chronic disease datasets and its performance was recorded. A very optimal and consistent classification performance was observed. The mean values for accuracy, specificity, sensitivity and f-score metrics were 96.7%, 96.5%, 95.6% and 96.2% respectively. [ABSTRACT FROM AUTHOR]

High dimensional biomedical data contain tens of thousands of features, accurate and effective identification of the core features in these data can be used to assist diagnose related diseases. However, there are often a large number of irrelevant or redundant features in biomedical data, which seriously affect subsequent classification accuracy and machine learning efficiency. To solve this problem, a novel filter feature selection algorithm based on redundant removal (FSBRR) is proposed to classify high dimensional biomedical data in this paper. First of all, two redundant criteria are determined by vertical relevance (the relationship between feature and class attribute) and horizontal relevance (the relationship between feature and feature). Secondly, to quantify redundant criteria, an approximate redundancy feature framework based on mutual information (MI) is defined to remove redundant and irrelevant features. To evaluate the effectiveness of our proposed algorithm, controlled trials based on typical feature selection algorithm are conducted using three different classifiers, and the experimental results indicate that the FSBRR algorithm can effectively reduce the feature dimension and improve the classification accuracy. In addition, an experiment of small sample dataset is designed and conducted in the section of discussion and analysis to clarify the specific implementation process of FSBRR algorithm more clearly. [ABSTRACT FROM AUTHOR]

There are numerous combinations of neural networks (NNs) and evolutionary algorithms (EAs) used in classification problems. EAs have been used to train the networks, design their architecture, and select feature subsets. However, most of these combinations have been tested on only a few data sets and many comparisons are done inappropriately measuring the performance on training data or without using proper statistical tests to support the conclusions. This paper presents an empirical evaluation of eight combinations of EAs and NNs on 15 public-domain and artificial data sets. Our objective is to identify the methods that consistently produce accurate classifiers that generalize well. In most cases, the combinations of EAs and NNs perform equally well on the data sets we tried and were not more accurate than hand-designed neural networks trained with simple backpropagation. [ABSTRACT FROM AUTHOR]

The sentiment classification method is a research field that is proliferating in Indonesia since it is fast in extracting public opinion and provides essential and valuable information for stakeholders. Of the best-performing sentiment classification approaches, machine learning is one of them that has excellent performance. However, the method has several problems, such as noisy features and high dimensionality of features that significantly affect the sentiment classification performance. Therefore, to overcome the problems, this paper presents a novel feature selection using a combination of Query Expansion Ranking (QER) and Genetic Algorithm-Support Vector Machine (GA-SVM) for improving sentiment classification performance. Based on the experimental results, the proposed method could significantly improve sentiment classification performance, outperform all state-of-the-art algorithms, and decrease computational time. The method achieved the best performance in average precision, recall, and f-measure with the value of 96.78%, 96.76%, and 96.75%, respectively. [ABSTRACT FROM AUTHOR]

Due to the insidious characteristics of network intrusion behaviors, developing an efficient intrusion detection system is still a big challenge, especially in the era of big data where the number of traffic and the dimension of each traffic feature are high. Because of the shortcomings of traditional common machine learning algorithms in network intrusion detection, such as insufficient accuracy, a network intrusion detection system based on LightGBM and autoencoder (AE) is proposed. The LightGBM-AE model proposed in this paper includes three steps: data preprocessing, feature selection, and classification. The LightGBM-AE model adopts the LightGBM algorithm for feature selection, and then uses an autoencoder for training and detection. When a set of data containing network intrusion behaviors are inputted into an autoencoder, there is a large reconstruction error between the original input data and the reconstructed data obtained by the autoencoder, which provides a basis for intrusion detection. According to the reconstruction error, an appropriate threshold is set to distinguish symmetrically between normal behavior and attack behavior. The experiment is carried out on the NSL-KDD dataset and implemented using Pytorch. In addition to autoencoder, variational autoencoder (VAE) and denoising autoencoder (DAE) are also used for intrusion detection and are compared with existing machine learning algorithms such as Decision Tree, Random Forest, KNN, GBDT, and XGBoost. The evaluation is carried out through classification evaluation indexes such as accuracy, precision, recall, F1-score. The experimental results show that the method can efficiently separate the attack behavior from normal behavior according to the reconstruction error. Compared with other methods, the effectiveness and superiority of this method are verified. [ABSTRACT FROM AUTHOR]

Electrocardiography (ECG) signals provides assistance to the cardiologists for identification of various cardiovascular diseases (CVD). ECG machine records the electrical activity of the heart with the assistance of electrodes placed on the patient's body. Qualitative characterization of ECG signal reflects its sensitiveness towards distinct artifacts that resulted in low diagnostic accuracy and may lead to incorrect decision of the clinician. The artifacts are removed utilizing a robust noise estimator employing DTCWT using various threshold values and functions. The segments and intervals of ECG signals are calculated using the peak detection algorithm followed by particle swarm optimization (PSO) and the proposed optimization technique to select the best features from a considerable pool of features. Out of the 12 features, the best four features are selected using PSO and the proposed optimization technique. Comparative analysis with other feature selection methods and state-of-the-art techniques demonstrated that the proposed algorithm precisely selects principle features for handling the ECG signal and attains better classification utilizing distinctive machine learning algorithms. The obtained accuracy using our proposed optimization technique is 95.71% employing k -NN and neural networks. Also, 4% and 10% improvements have been observed while using k -NN over ANN and SVM, respectively, when the PSO technique is executed. Similarly, a 14.16% improvement is achieved while using k -NN and ANN over the SVM machine learning technique for the proposed optimization technique. Heart rate is calculated using the proposed estimator and optimization technique, which is in consensus with the gold standard. [ABSTRACT FROM AUTHOR]

Reducing the number of features when applying machine learning algorithms may be beneficial not only from the standpoint of computational cost but also of overall quality. Wrapper-based procedures are widely utilised to achieve this. The choice of the wrapper is of utmost importance. Bio-inspired computation algorithms represent a viable choice and are widely adopted. Due to the sheer number of available algorithms, this choice could prove to be somewhat difficult, especially since not all are made equally. The aim of this paper is to explore several optimisers on diverse datasets representing classification problems in order to evaluate their performance and suitability for the task of feature selection. [ABSTRACT FROM AUTHOR]

The advent of modern genotyping technologies has revolutionized genomic selection in animal breeding. Large marker datasets have shown several drawbacks for traditional genomic prediction methods in terms of flexibility, accuracy, and computational power. Recently, the application of machine learning models in animal breeding has gained a lot of interest due to their tremendous flexibility and their ability to capture patterns in large noisy datasets. Here, we present a general overview of a handful of machine learning algorithms and their application in genomic prediction to provide a meta-picture of their performance in genomic estimated breeding values estimation, genotype imputation, and feature selection. Finally, we discuss a potential adoption of machine learning models in genomic prediction in developing countries. The results of the reviewed studies showed that machine learning models have indeed performed well in fitting large noisy data sets and modeling minor nonadditive effects in some of the studies. However, sometimes conventional methods outperformed machine learning models, which confirms that there's no universal method for genomic prediction. In summary, machine learning models have great potential for extracting patterns from single nucleotide polymorphism datasets. Nonetheless, the level of their adoption in animal breeding is still low due to data limitations, complex genetic interactions, a lack of standardization and reproducibility, and the lack of interpretability of machine learning models when trained with biological data. Consequently, there is no remarkable outperformance of machine learning methods compared to traditional methods in genomic prediction. Therefore, more research should be conducted to discover new insights that could enhance livestock breeding programs. [ABSTRACT FROM AUTHOR]

Simple Summary: Glioma tumor aggressiveness is expressed as tumor grading which is crucial in guiding treatment decisions and clinical trial participation. Accurate and standardized grading systems are essential to optimize care and improve outcomes. However, integrating molecular and clinical information in the grading process has the potential to expose molecular markers that have gained importance in understanding tumor biology as a means of identifying druggable targets. In this study, a novel approach called GradWise is introduced with the goal of enhancing feature selection performance while employing various machine learning models of glioma grading. GradWise combines a rank-based weighted hybrid filter (mRMR) and an embedded feature selection method (LASSO) to select the most relevant features from clinical and molecular predictors and was evaluated using two commonly employed public biomedical datasets, TCGA and CGGA, utilizing two feature selection methods and five supervised models. The findings support existing evidence and provide pioneering results for glioma-specific biomarkers, highlighting the effectiveness of the approach and future directions for biological mechanisms of glioma progression to higher grades. Glioma grading plays a pivotal role in guiding treatment decisions, predicting patient outcomes, facilitating clinical trial participation and research, and tailoring treatment strategies. Current glioma grading in the clinic is based on tissue acquired at the time of resection, with tumor aggressiveness assessed from tumor morphology and molecular features. The increased emphasis on molecular characteristics as a guide for management and prognosis estimation underscores is driven by the need for accurate and standardized grading systems that integrate molecular and clinical information in the grading process and carry the expectation of the exposure of molecular markers that go beyond prognosis to increase understanding of tumor biology as a means of identifying druggable targets. In this study, we introduce a novel application (GradWise) that combines rank-based weighted hybrid filter (i.e., mRMR) and embedded (i.e., LASSO) feature selection methods to enhance the performance of feature selection and machine learning models for glioma grading using both clinical and molecular predictors. We utilized publicly available TCGA from the UCI ML Repository and CGGA datasets to identify the most effective scheme that allows for the selection of the minimum number of features with their names. Two popular feature selection methods with a rank-based weighting procedure were employed to conduct comprehensive experiments with the five supervised models. The computational results demonstrate that our proposed method achieves an accuracy rate of 87.007% with 13 features and an accuracy rate of 80.412% with five features on the TCGA and CGGA datasets, respectively. We also obtained four shared biomarkers for the glioma grading that emerged in both datasets and can be employed with transferable value to other datasets and data-based outcome analyses. These findings are a significant step toward highlighting the effectiveness of our approach by offering pioneering results with novel markers with prospects for understanding and targeting the biologic mechanisms of glioma progression to improve patient outcomes. [ABSTRACT FROM AUTHOR]

Coronary artery disease (CAD) is one of the diseases with the highest morbidity and mortality in the world. In 2019, the number of deaths caused by CAD reached 9.14 million. The detection and treatment of CAD in the early stage is crucial to save lives and improve prognosis. Therefore, the purpose of this research is to develop a machine-learning system that can be used to help diagnose CAD accurately in the early stage. In this paper, two classical ensemble learning algorithms, namely, XGBoost algorithm and Random Forest algorithm, were used as the classification model. In order to improve the classification accuracy and performance of the model, we applied four feature processing techniques to process features respectively. In addition, synthetic minority oversampling technology (SMOTE) and adaptive synthetic (ADASYN) were used to balance the dataset, which included 71.29% CAD samples and 28.71% normal samples. The four feature processing technologies improved the performance of the classification models in terms of classification accuracy, precision, recall, F1 score and specificity. In particular, the XGBboost algorithm achieved the best prediction performance results on the dataset processed by feature construction and the SMOTE method. The best classification accuracy, recall, specificity, precision, F1 score and AUC were 94.7%, 96.1%, 93.2%, 93.4%, 94.6% and 98.0%, respectively. The experimental results prove that the proposed method can accurately and reliably identify CAD patients from suspicious patients in the early stage and can be used by medical staff for auxiliary diagnosis. [ABSTRACT FROM AUTHOR]

Feature selection is a crucial process in machine learning and data mining that identifies the most pertinent and valuable features in a dataset. It enhances the efficacy and precision of predictive models by efficiently reducing the number of features. This reduction improves classification accuracy, lessens the computational burden, and enhances overall performance. This study proposes the improved binary golden jackal optimization (IBGJO) algorithm, an extension of the conventional golden jackal optimization (GJO) algorithm. IBGJO serves as a search strategy for wrapper-based feature selection. It comprises three key factors: a population initialization process with a chaotic tent map (CTM) mechanism that enhances exploitation abilities and guarantees population diversity, an adaptive position update mechanism using cosine similarity to prevent premature convergence, and a binary mechanism well-suited for binary feature selection problems. We evaluated IBGJO on 28 classical datasets from the UC Irvine Machine Learning Repository. The results show that the CTM mechanism and the position update strategy based on cosine similarity proposed in IBGJO can significantly improve the Rate of convergence of the conventional GJO algorithm, and the accuracy is also significantly better than other algorithms. Additionally, we evaluate the effectiveness and performance of the enhanced factors. Our empirical results show that the proposed CTM mechanism and the position update strategy based on cosine similarity can help the conventional GJO algorithm converge faster. [ABSTRACT FROM AUTHOR]

A number of different feature selection and classification techniques have been proposed in literature including parameter-free and parameter-based algorithms. The former are quick but may result in local maxima while the latter use dataset-specific parameter-tuning for higher accuracy. However, higher accuracy may not necessarily mean higher reliability of the model. Thus, generalized optimization is still a challenge open for further research. This paper presents a warzone inspired "infiltration tactics" based optimization algorithm (ITO)—not to be confused with the ITO algorithm based on the Itõ Process in the field of Stochastic calculus. The proposed ITO algorithm combines parameter-free and parameter-based classifiers to produce a high-accuracy-high-reliability (HAHR) binary classifier. The algorithm produces results in two phases: (i) Lightweight Infantry Group (LIG) converges quickly to find non-local maxima and produces comparable results (i.e., 70 to 88% accuracy) (ii) Followup Team (FT) uses advanced tuning to enhance the baseline performance (i.e., 75 to 99%). Every soldier of the ITO army is a base model with its own independently chosen Subset selection method, pre-processing, and validation methods and classifier. The successful soldiers are combined through heterogeneous ensembles for optimal results. The proposed approach addresses a data scarcity problem, is flexible to the choice of heterogeneous base classifiers, and is able to produce HAHR models comparable to the established MAQC-II results. [ABSTRACT FROM AUTHOR]

The wrapper algorithm adopts the performance of the learning algorithm as the evaluation criteria to obtain excellent classification performance. However, the wrapper algorithm is prone to converge prematurely. A global chaotic bat algorithm (GCBA) is put up forward to improve this shortage. First, GCBA applies chaotic map to population initialization to cover the entire solution space. In addition, adaptive learning factors are presented to balance exploration and exploration. The learning factor of local optimal position gradually decreases in the early stage while the learning factor of global optimal position gradually increases in the later stage. Finally, to improve the exploitation, an improved transfer function is proposed, which transfers the continuous space to discrete binary space. GCBA is tested on 14 UCI data sets and 5 gene expression data sets compared with other 6 comparison algorithms. Compared with other algorithms, the results show that GCBA is able to achieve better classification performance. [ABSTRACT FROM AUTHOR]

This research aims to analyze the effect of feature selection on the accuracy of music popularity classification using machine learning algorithms. The data of Spotify, the most used music listening platform today, was used in the research. In the feature selection stage, features with low correlation were removed from the dataset using the filter feature selection method. Machine learning algorithms using all features produced 95.15% accuracy, while machine learning algorithms using features selected by feature selection produced 95.14% accuracy. The features selected by feature selection were sufficient for classification of popularity in established algorithms. In addition, this dataset contains fewer features, so the computation time is shorter. The reason why Big O time complexity is lower than models constructed without feature selection is that the number of features, which is the most important parameter in time complexity, is low. The statistical analysis was performed on the pre-processed data and meaningful information was produced from the data using machine learning algorithms. [ABSTRACT FROM AUTHOR]

Machine learning algorithms are widely applied to biomedical data to classify the samples of patients and healthy persons. The high‐dimensional biomedical datasets contain a large number of features to represent a sample. However, such datasets may have redundant, noisy and irrelevant features, influencing machine learning algorithms' classification performance and increasing computation overhead. Therefore, data normalization and feature selection techniques are introduced to reduce the impact of noisy features and accurately identify the patterns in features to improve the predictive accuracy of diagnosis. This work proposes an efficient feature selection and parameter optimization method to classify high‐dimensional biomedical datasets. In the proposed method, a binary version of the improved elephant herding optimization (IEHO) algorithm is introduced to select features and optimize the C and γ parameters of the support vector machine classifier. Further, four variants of the proposed method are presented based on data normalization techniques: Z‐score normalization (ZN), Pareto‐scaling (PS), tan h‐based normalization (TN), and variant of tan h‐based normalization (VTN). The proposed variants reduce the dominance of noisy features and explore the feature space to obtain the optimal feature set that maximizes the classification accuracy and minimizes the time complexity. The performance of the proposed variants is evaluated on 15 high‐dimensional biomedical datasets. Friedman's mean rank test is applied to check the statistical difference between proposed variants. Results show that the proposed Z‐score normalization‐IEHO (ZN‐IEHO) variant performed significantly better than the other proposed variants for classification accuracy, false‐positive rate and f‐score metrics. Moreover, the performance of the proposed ZN‐IEHO variant is compared with 18 state‐of‐the‐art feature selection methods. The experimental results expressed the effectiveness of the proposed ZN‐IEHO variant in finding the best combination of features and parameters to classify the biomedical datasets accurately. [ABSTRACT FROM AUTHOR]

With the advent of Big Data, data is being collected at an unprecedented fast pace, and it needs to be processed in a short time. To deal with data streams that flow continuously, classical batch learning algorithms cannot be applied and it is necessary to employ online approaches. Online learning consists of continuously revising and refining a model by incorporating new data as they arrive, and it allows important problems such as concept drift or management of extremely high-dimensional datasets to be solved. In this paper, we present a unified pipeline for online learning which covers online discretization, feature selection and classification. Three classical methods—the k -means discretizer, the χ 2 filter and a one-layer artificial neural network—have been reimplemented to be able to tackle online data, showing promising results on both synthetic and real datasets. [ABSTRACT FROM AUTHOR]

Certain small noncoding microRNAs (miRNAs) are differentially expressed in normal tissues and cancers, which makes them great candidates for biomarkers for cancer. Previously, a selected subset of miRNAs has been experimentally verified to be linked to breast cancer. In this paper, we validated the importance of these miRNAs using a machine learning approach on miRNA expression data. We performed feature selection, using Information Gain (IG), Chi-Squared (CHI2) and Least Absolute Shrinkage and Selection Operation (LASSO), on the set of these relevant miRNAs to rank them by importance. We then performed cancer classification using these miRNAs as features using Random Forest (RF) and Support Vector Machine (SVM) classifiers. Our results demonstrated that the miRNAs ranked higher by our analysis had higher classifier performance. Performance becomes lower as the rank of the miRNA decreases, confirming that these miRNAs had different degrees of importance as biomarkers. Furthermore, we discovered that using a minimum of three miRNAs as biomarkers for breast cancers can be as effective as using the entire set of 1800 miRNAs. This work suggests that machine learning is a useful tool for functional studies of miRNAs for cancer detection and diagnosis. [ABSTRACT FROM AUTHOR]

The COVID‐19 pandemic has huge effects on the global community and an extreme burden on health systems. There are more than 185 million confirmed cases and 4 million deaths as of July 2021. Besides, the exponential rise in COVID‐19 cases requires a quick prediction of the patients' severity for better treatment. In this study, we propose a Multi‐threaded Genetic feature selection algorithm combined with Extreme Learning Machines (MG‐ELM) to predict the severity level of the COVID‐19 patients. We conduct a set of experiments on a recently published real‐world dataset. We reprocess the dataset via feature construction to improve the learning performance of the algorithm. Upon comprehensive experiments, we report the most impactful features and symptoms for predicting the patients' severity level. Moreover, we investigate the effects of multi‐threaded implementation with statistical analysis. In order to verify the efficiency of MG‐ELM, we compare our results with traditional and state‐of‐the‐art techniques. The proposed algorithm outperforms other algorithms in terms of prediction accuracy. [ABSTRACT FROM AUTHOR]

In this article, a hybrid algorithm has been proposed for the identification of phishing and legitimate websites. The dataset may have an imbalanced class distribution and may consist of irrelevant features. Therefore, in the data preprocessing, the adaptive synthetic sampling approach has been used to handle the imbalanced data. Irrelevant or redundant features are removed from the balanced data using the proposed binary version of Rao algorithms. The S‐shaped and V‐shaped transfer functions are applied for mapping continuous search space to discrete search space. Also, the results of these S‐shaped and V‐shaped transfer functions are analyzed for proposed algorithms. The performance is improved by optimizing the value of the k parameter in the kNN classifier. The dataset used in this article has been taken from the UCI machine‐learning repository. The performance of the proposed approach has been evaluated using the polygon area metric. The obtained classification accuracy is 97.044%. A comparison of the proposed hybrid algorithm with the other state‐of‐the‐art techniques is also made for validation. Moreover, the proposed approach has been compared with seven metaheuristic feature selection algorithms and six filter methods for performance analysis. Additionally, we have applied the proposed approach to URLs that are registered on the PhishTank website. [ABSTRACT FROM AUTHOR]

Sine Cosine Algorithm (SCA) that is one of the population-based metaheuristic optimization algorithms basically consists of the updating mechanism based on sine and cosine functions. In this algorithm, a few random and adaptive variables are also utilized for more effective motions of the candidate solutions. SCA has some drawbacks like other some metaheuristic algorithms. SCA tends to be stuck into the local regions in the search space and this affects negatively on the computational effort required to find the best solution point in the search space. This paper presents four different improved versions of SCA. The proposed improvements on original SCA are the innovations on the updating mechanism of SCA. To evaluate the performances of Improved Sine Cosine Algorithms (ImpSCAs), well-known numerical optimization problems including CEC 2014 test suite are used. Firstly, different analyses of the proposed ImpSCAs are dealt with such as the convergence analysis, search history analysis, trajectory analysis, average distance analysis, and computational complexity analysis. Secondly, the proposed four versions of ImpSCAs are compared with the original SCA for CEC 2014 benchmark problems with dimension sizes of 10D, 30D and 50D. Finally, original SCA and ImpSCAs are adapted to select optimal feature combination and they are tested for 10 feature selection datasets taken from the UCI machine learning repository. The benchmark results show that the performances of the ImpSCA 1 , ImpSCA 2 , and ImpSCA 4 are better than that of the original SCA. From the feature selection results, it is observed that three versions of ImpSCAs (except ImpSCA 3) outperform the original SCA in 80% of the datasets. Source codes of ImpSCAs are publicly available at https://github.com/uguryuzgec/ImpSCAs. • We focused to improve the exploration and exploitation ability of the original SCA. • The basic idea is based on the innovations on the update mechanism. • The ISCAs were evaluated using CEC2014 benchmark test suite. • The proposed ISCAs were tested for 10 popular UCI datasets. • Three versions of ISCAs (except ISCA 3) outperform the SCA in 80% of the datasets. [ABSTRACT FROM AUTHOR]

Abstract: Feature selection and classification techniques have been studied independently without considering the interaction between both procedures, which leads to a degraded performance. In this paper, we present a new neural network approach, which is called an algorithm learning based neural network (ALBNN), to improve classification accuracy by integrating feature selection and classification procedures. In general, a knowledge-based artificial neural network operates on prior knowledge from domain experience, which provides it with better starting points for the target function and leads to better classification accuracy. However, prior knowledge is usually difficult to identify. Instead of using unknown background resources, the proposed method utilizes prior knowledge that is mathematically calculated from the properties of other learning algorithms such as PCA, LARS, C4.5, and SVM. We employ the extreme learning machine in this study to help obtain better initial points faster and avoid irrelevant time-consuming work, such as determining architecture and manual tuning. ALBNN correctly approximates a target hypothesis by both considering the interaction between two procedures and minimizing individual procedure errors. The approach produces new relevant features and improves the classification accuracy. Experimental results exhibit improved performance in various classification problems. ALBNN can be applied to various fields requiring high classification accuracy. [Copyright &y& Elsevier]

Abstract: Feature selection is one of the core issues in designing pattern recognition and machine learning systems, and has attracted considerable attention in the literature. In this paper, a new feature subset selection algorithm with conditional mutual information is proposed, which firstly guarantees to find a subset of which the mutual information with the class is the same as that of the original set of features, and then eliminates potential redundant features from the view of minimal information loss based on the cumulate conditional mutual information minimization criterion. From the reliability point of view, this criterion can also abate the disturbance caused by sample insufficiency in conditional mutual information estimation. In addition, a fast implementation of conditional mutual information estimation is proposed and used to tackle the computationally intractable problem. Empirical results verify that our algorithm is efficient and achieves better accuracy than several representative feature selection algorithms for three typical classifiers on various datasets. [Copyright &y& Elsevier]

This paper presents new and effective algorithms for learning kernels. In particular, as shown by our empirical results, these algorithms consistently outperform the so-called uniform combination solution that has proven to be difficult to improve upon in the past, as well as other algorithms for learning kernels based on convex combinations of base kernels in both classification and regression. Our algorithms are based on the notion of centered alignment which is used as a similarity measure between kernels or kernel matrices. We present a number of novel algorithmic, theoretical, and empirical results for learning kernels based on our notion of centered alignment. In particular, we describe efficient algorithms for learning a maximum alignment kernel by showing that the problem can be reduced to a simple QP and discuss a one-stage algorithm for learning both a kernel and a hypothesis based on that kernel using an alignment-based regularization. Our theoretical results include a novel concentration bound for centered alignment between kernel matrices, the proof of the existence of effective predictors for kernels with high alignment, both for classification and for regression, and the proof of stability-based generalization bounds for a broad family of algorithms for learning kernels based on centered alignment. We also report the results of experiments with our centered alignment-based algorithms in both classification and regression. [ABSTRACT FROM AUTHOR]

Abstract: In order to clarify the role of AdaBoost algorithm for feature selection, classifier learning and its relation with SVM, this paper provided a brief introduction to the AdaBoost which is used for producing a strong classifier out of weak learners firstly. The original adaptive boosting algorithm and its application in face detection and facial expression recognition are reviewed. In pattern classification domain, support vector machine has been widely used and shows promising performance. However, it is expensive in terms of time-consuming. A sort of cascaded support vector machines architecture is capable of improving the classification accuracy based on AdaBoost boosting algorithm, namely, AdaboostSVM. It applied boosting algorithm to feature selection and classifier learning for support vector machine classification and it has achieved approved performance through some researcher''s pioneering work. [Copyright &y& Elsevier]

Abstract: The inclusion of irrelevant, redundant, and inconsistent features in the data-mining model results in poor predictions and high computational overhead. This paper proposes a novel information theoretic-based interact (IT-IN) algorithm, which concerns the relevance, redundancy, and consistency of the features. The proposed IT-IN algorithm is compared with existing Interact, FCBF, Relief and CFS feature selection algorithms. To evaluate the classification accuracy of IT-IN and remaining four feature selection algorithms, Naïve Bayes, SVM, and ELM classifier are used for ten UCI repository datasets. The proposed IT-IN performs better than existing above algorithms in terms of number of features. The specially designed hash function is used to speed up the IT-IN algorithms and provides minimum computation time than the Interact algorithms. The result clearly reveals that the proposed feature selection algorithm improves the classification accuracy for ELM, Naïve Bayes, and SVM classifiers. The performance of proposed IT-IN with ELM classifier is superior to other classifiers. [ABSTRACT FROM AUTHOR]

Abstract: There are irrelevant features that are redundant or significantly degrade the learning accuracy in the real-world complex classification tasks. This paper presents a new hybrid learning algorithm based on a cooperative coevolutionary algorithm (Co-CEA) with dual populations for designing the radial basis function neural network (RBFNN) models with an explicit feature selection. This approach attempts to complete both the RBFNN construction and the feature selection simultaneously. The proposed algorithm utilizes the Co-CEA’s divide-and-cooperative mechanism, which utilizes the evolutionary algorithms executing in parallel to coevolve subpopulations, corresponding to the hidden layer structure and the dominate features respectively. The algorithm adopts the binary encoding to represent the feature subset and the matrix-form mixed encoding to represent the RBFNN hidden layer structure, and a complete solution is formed via collaborations among the two subpopulations. Experimental results illustrate that the proposed algorithm outperforms other algorithms in references in terms of the classification accuracy, and it is able to obtain both prominent features and good RBFNN structure with higher prediction capability. [Copyright &y& Elsevier]

Abstract: Algorithms for feature selection in predictive data mining for classification problems attempt to select those features that are relevant, and are not redundant for the classification task. A relevant feature is defined as one which is highly correlated with the target function. One problem with the definition of feature relevance is that there is no universally accepted definition of what it means for a feature to be ‘highly correlated with the target function or highly correlated with the other features’. A new feature selection algorithm which incorporates domain specific definitions of high, medium and low correlations is proposed in this paper. The proposed algorithm conducts a heuristic search for the most relevant features for the prediction task. [Copyright &y& Elsevier]

Simple Summary: Radiomics aims to extract high-dimensional features from clinical images and associate them to clinical outcomes. These associations may be further investigated with machine learning models; however, guidelines on the most suitable method to support clinical decisions are still missing. To improve the reliability and the accuracy of radiomic features in the prediction of a binary variable in a lung cancer setting, we compared several machine learning classifiers and feature selection methods on simulated data. These account for important characteristics that may vary in real clinical datasets: sample size, outcome balancing and association strength between radiomic features and outcome variables. We were able to suggest the most suitable classifiers for each studied case and to evaluate the impact of method choices. Our work highlights the importance of these choices in radiomic analyses and provides guidelines on how to select the best models for the data at hand. Radiomics uses high-dimensional sets of imaging features to predict biological characteristics of tumors and clinical outcomes. The choice of the algorithm used to analyze radiomic features and perform predictions has a high impact on the results, thus the identification of adequate machine learning methods for radiomic applications is crucial. In this study we aim to identify suitable approaches of analysis for radiomic-based binary predictions, according to sample size, outcome balancing and the features–outcome association strength. Simulated data were obtained reproducing the correlation structure among 168 radiomic features extracted from Computed Tomography images of 270 Non-Small-Cell Lung Cancer (NSCLC) patients and the associated to lymph node status. Performances of six classifiers combined with six feature selection (FS) methods were assessed on the simulated data using AUC (Area Under the Receiver Operating Characteristics Curves), sensitivity, and specificity. For all the FS methods and regardless of the association strength, the tree-based classifiers Random Forest and Extreme Gradient Boosting obtained good performances (AUC ≥ 0.73), showing the best trade-off between sensitivity and specificity. On small samples, performances were generally lower than in large–medium samples and with larger variations. FS methods generally did not improve performances. Thus, in radiomic studies, we suggest evaluating the choice of FS and classifiers, considering specific sample size, balancing, and association strength. [ABSTRACT FROM AUTHOR]

Background: The small number of samples and the curse of dimensionality hamper the better application of deep learning techniques for disease classification. Additionally, the performance of clustering-based feature selection algorithms is still far from being satisfactory due to their limitation in using unsupervised learning methods. To enhance interpretability and overcome this problem, we developed a novel feature selection algorithm. In the meantime, complex genomic data brought great challenges for the identification of biomarkers and therapeutic targets. The current some feature selection methods have the problem of low sensitivity and specificity in this field. Results: In this article, we designed a multi-scale clustering-based feature selection algorithm named MCBFS which simultaneously performs feature selection and model learning for genomic data analysis. The experimental results demonstrated that MCBFS is robust and effective by comparing it with seven benchmark and six state-of-the-art supervised methods on eight data sets. The visualization results and the statistical test showed that MCBFS can capture the informative genes and improve the interpretability and visualization of tumor gene expression and single-cell sequencing data. Additionally, we developed a general framework named McbfsNW using gene expression data and protein interaction data to identify robust biomarkers and therapeutic targets for diagnosis and therapy of diseases. The framework incorporates the MCBFS algorithm, network recognition ensemble algorithm and feature selection wrapper. McbfsNW has been applied to the lung adenocarcinoma (LUAD) data sets. The preliminary results demonstrated that higher prediction results can be attained by identified biomarkers on the independent LUAD data set, and we also structured a drug-target network which may be good for LUAD therapy. Conclusions: The proposed novel feature selection method is robust and effective for gene selection, classification, and visualization. The framework McbfsNW is practical and helpful for the identification of biomarkers and targets on genomic data. It is believed that the same methods and principles are extensible and applicable to other different kinds of data sets. [ABSTRACT FROM AUTHOR]