Search

• A feature selection method based on binary dandelion algorithm is proposed. • The algorithm applies a sine-cosine operator and a restart strategy. • Mutual information and quick bit mutation improve the performance of the algorithm. • The algorithm performs well on datasets with larger dimensions. Feature selection (FS) is an important data preprocessing technology for machine learning and data mining. Metaheuristic algorithm (MH) has been widely used in feature selection because of its powerful search function. This paper presents an improved Binary Dandelion Algorithm using Sine Cosine operator and Restart strategy (SCRBDA) for feature selection. First, the sine cosine operator is used in the radius formula of the core dandelions (CD), which significantly enhances the ability of algorithm development and exploration. Secondly, the algorithm uses a restart strategy to increase its ability to get rid of local optimum. Thirdly, mutual information is used to guide the generation of some dandelions, which pays more attention to the correlation between the selected features and categories. Finally, quick bit mutation is used as the mutation strategy to improve the diversity of the population. The SCRBDA proposed in this paper was tested on 18 datasets of different sizes from UCI machine learning database. The SCRBDA was compared with 8 other classical feature selection algorithms, and the performance of the proposed algorithm was evaluated through feature subset size, classification accuracy, fitness value, and F1-score. The experimental results show that SCRBDA achieves the best performance, which has stronger feature reduction ability and achieves better overall performance on most datasets. Especially on large-scale datasets, SCRBDA can obtain extremely smaller feature subsets while maintaining much higher classification accuracy, and satisfactory F1-score. [ABSTRACT FROM AUTHOR]

• MMODE is developed as a new hybrid gene selection method for tumor identification. • MMODE combines multi-tasking and multi-objective frameworks. • MMODE uses a new elite strategy and a new guidance strategy. • MMODE selects a few genes and achieves high classification accuracy. A key preprocessing step in tumor recognition based on microarray expression profile data and machine learning is to identify tumor marker genes. Gene selection aims to select the most relevant gene subset from the original ultra-high dimensional microarray expression profile data to improve tumor identification performance. Inspired by evolutionary multitasking (EMT) and multi-objective optimization, this paper puts forward a novel multitasking multi-objective differential evolution gene selection algorithm (MMODE) which uses new elite and guidance strategies to select the best gene subsets. MMODE initializes two different populations according to different filtering criteria to increase the diversity of the search. These two populations guide their respective populations to search in the optimal direction through knowledge transfer in the evolutionary process. In addition, MMODE employs new elite and guidance strategies that enables individuals to narrow the search range and jump out of local optima. The proposed algorithm is validated on 13 publicly available microarray expression datasets in comparison with state-of-the-art gene selection algorithms. The experimental results show that MMODE can find smaller gene subsets and achieve higher classification accuracy compared with other algorithms. [ABSTRACT FROM AUTHOR]

Attribute reduction can remove data noise and redundancy, thus reducing computational complexity, which is very important for machine learning. Because the difference between nominal attribute values is difficult to measure, attribute reduction for hybrid data faces challenges. In addition, most of the existing methods are sensitive to noise due to the lack of an anti-noise mechanism. Decision attribute contains the most important information of data. This paper proposes some techniques that consider the above problems from the perspective of fuzzy evidence theory. First of all, a new distance incorporating decision attributes is defined, and then a new fuzzy relation with an anti-noise mechanism is defined. Furthermore, fuzzy belief and fuzzy plausibility are defined based on the defined new distance and new fuzzy relation. In this framework, two anti-noise attribute reduction algorithms for hybrid data are proposed. Experiments on 12 data sets of various types show that compared with the other 8 state-of-the-art algorithms, the proposed algorithms improve the classification accuracy by at least 2% and the anti-noise ability by at least 11%. Therefore, it can be concluded that the proposed algorithms have excellent anti-noise ability while maintaining good feature selection ability. [ABSTRACT FROM AUTHOR]

Age estimation has become inordinately significant for human beings for many reasons, such as detecting legal and criminal responsibility and other social events like a marriage license, birth certificate, etc. This paper aims to decide on the most desirable machine learning algorithm (from conventional machine learning algorithms to deep learning) for dental age estimation based on buccal bone level. The database consisted of 150 CBCT images (73 males and 77 females) from an existing base of the Faculty of Dental Medicine with Clinics, University of Sarajevo, aged 20-69. Results were obtained using the Waikato Environment for Knowledge Analysis (Weka), machine learning software in Java. Left and Right Buccal Alveolar Bone Levels are increasing with age, so they showed to be the most important attributes, especially the latter. Random Forest classifier provided the greatest result with the correlation coefficient of 0.803 and the mean absolute error of 6.022. We have also shown that considering sinus-related features can be a significant addition to the databases. Our paper is probably one of the first studies where regression algorithms based on the Support Vector Machines and Random Forest were utilized. [ABSTRACT FROM AUTHOR]

• 3 layer neural networks outperform other ML algorithms in automated assessments. • In subjective assignments, deep neural networks can improve score prediction. • Word embeddings and linguistic feature results in more precise text mining models. • Extraction and selection of appropriate features can improve results. Machine learning (ML) has recently gained popularity in a variety of domains for automating tasks. This has also been used in the evaluation of student responses/ assignments. This paper presents a qualitatively enhanced methodology for automated score prediction of subjective assignments. To consider all the major aspects of a human grader, 21 linguistic features related to syntactical, grammatical, sentimental, and readability are analyzed from the assignments. This study makes use of the ASAP competition dataset from Kaggle. The evaluation metric used is Quadratic Weighted Kappa (QWK), which measures the agreement between the human graded score and the predicted score. The effect of appropriate feature selection has been observed using Mutual Information Regression. Four ML algorithms are investigated on identified linguistic features. 3 Layer Neural Network with feature selection performed well among all chosen ML algorithms with average QWK of 0.678. To include the benefits of deep learning, a new hybrid model (LF-BiLSTM-att-FS) is proposed that combines a higher level deep neural network (DNN) with the selected features. Pre-trained Glove embedding is used to include contextual information of the text. The proposed model results demonstrated an improvement in overall accuracy, with an average QWK value of 0.768. [ABSTRACT FROM AUTHOR]

Projected gradient descent has been proved efficient in many optimization and machine learning problems. The weighted ℓ 1 ball has been shown effective in sparse system identification and features selection. In this paper we propose three new efficient algorithms for projecting any vector of finite length onto the weighted ℓ 1 ball. The first two algorithms have a linear worst case complexity. The third one has a highly competitive performances in practice but the worst case has a quadratic complexity. These new algorithms are efficient tools for machine learning methods based on projected gradient descent such as compressed sensing, feature selection. We illustrate this effectiveness by adapting an efficient compressed sensing algorithm to weighted projections. We demonstrate the efficiency of our new algorithms on benchmarks using very large vectors. For instance, it requires only 8 ms, on an Intel I7 3rd generation, for projecting vectors of size 107. [ABSTRACT FROM AUTHOR]

Feature selection (FS) is recognized as one of the majority public and challenging problems in the Machine Learning domain. FS can be examined as an optimization problem that needs an effective optimizer to determine its optimal subset of more informative features. This paper proposes a wrapper FS method that combines chaotic maps (CMs) and binary Group Search Optimizer (GSO) called CGSO, which is used to solve the FS problem. In this method, five chaotic maps are incorporated with the GSO algorithm's main procedures, namely, Logistic, Piecewise, Singer, Sinusoidal, and Tent. The GSO algorithm is used as a search strategy, while k-NN is employed as an induction algorithm. The objective function is to integrate three main objectives: maximizing the classification accuracy value, minimizing the number of selected features, and minimizing the complexity of generated k-NN models. To evaluate the proposed methods' performance, twenty well-known UCI datasets are used and compared with other well-known published methods in the literature. The obtained results reveal the superiority of the proposed methods in outperforming other well-known methods, especially when using binary GSO with Tent CM. Finally, it is a beneficial method to be utilized in systems that require FS pre-processing. • A novel feature selection method is proposed for classification task. • Five Chaotic Group Search Optimizers are investigated. • Twenty UCI datasets were used in the experiments. • Our methods got a higher accuracy results using a fewer number of features. [ABSTRACT FROM AUTHOR]

Sine Cosine Algorithm (SCA) that is one of the population-based metaheuristic optimization algorithms basically consists of the updating mechanism based on sine and cosine functions. In this algorithm, a few random and adaptive variables are also utilized for more effective motions of the candidate solutions. SCA has some drawbacks like other some metaheuristic algorithms. SCA tends to be stuck into the local regions in the search space and this affects negatively on the computational effort required to find the best solution point in the search space. This paper presents four different improved versions of SCA. The proposed improvements on original SCA are the innovations on the updating mechanism of SCA. To evaluate the performances of Improved Sine Cosine Algorithms (ImpSCAs), well-known numerical optimization problems including CEC 2014 test suite are used. Firstly, different analyses of the proposed ImpSCAs are dealt with such as the convergence analysis, search history analysis, trajectory analysis, average distance analysis, and computational complexity analysis. Secondly, the proposed four versions of ImpSCAs are compared with the original SCA for CEC 2014 benchmark problems with dimension sizes of 10D, 30D and 50D. Finally, original SCA and ImpSCAs are adapted to select optimal feature combination and they are tested for 10 feature selection datasets taken from the UCI machine learning repository. The benchmark results show that the performances of the ImpSCA 1 , ImpSCA 2 , and ImpSCA 4 are better than that of the original SCA. From the feature selection results, it is observed that three versions of ImpSCAs (except ImpSCA 3) outperform the original SCA in 80% of the datasets. Source codes of ImpSCAs are publicly available at https://github.com/uguryuzgec/ImpSCAs. • We focused to improve the exploration and exploitation ability of the original SCA. • The basic idea is based on the innovations on the update mechanism. • The ISCAs were evaluated using CEC2014 benchmark test suite. • The proposed ISCAs were tested for 10 popular UCI datasets. • Three versions of ISCAs (except ISCA 3) outperform the SCA in 80% of the datasets. [ABSTRACT FROM AUTHOR]

• FCS is proposed as a complementary step for principal component analysis. • PCA-FCS identifies the most relevant components based on their level of contribution. • PCA-FCS simultaneously performs dimension reduction and component selection. • PCA-FCS has substantially improved the accuracy of the random forest algorithm. Principal component analysis (PCA) is one the most effective and widely used dimensionality-reduction techniques which aggregates the maximal variance in the first few components. In many applications, the first few components of PCA are used in place of the original variables in statistical and machine learning models; nevertheless, such use does not guarantee the selection of the most relevant components to the target variable. This paper presents an efficient approach in which all components, rather than simply the first few, are considered as input for the random forest (RF) model, and a feature selection algorithm is integrated with the RF to select the most relevant components, hence called fully component selection (FCS). The proposed method was evaluated on spectroscopic data comprising 70 soil samples with 2050 spectral bands to estimate soil organic carbon (SOC). The FCS approach was compared to the RF model once considering the original features, then again using only the first few components, and again using all components without feature selection. The results showed that RF-FCS substantially outperformed other approaches, such that the R2 was increased between 25.2% and 55.5%. Furthermore, the findings indicated that the most relevant principal components (PCs) to the target variable were not the first few, but PC6, PC7, PC8, PC15, and PC43. Using the FCS approach increased model performance substantively, and coupling it with machine learning and statistical models is strongly recommended for high dimensional data applications. [ABSTRACT FROM AUTHOR]

Feature selection in real-world data mining problems is essential to make the learning task efficient and more accurate. Identifying the best feature selection algorithm, among the many available, is a complex activity that still relies heavily on human experts or some random trial-and-error procedure. Thus, the automated machine learning community has taken some steps towards the automation of this process. In this paper, we address the metalearning challenge of recommending feature selection algorithms by proposing a novel meta-feature engineering model. Our model considers a broad collection of meta-features that enable the study of the relationship between the dataset properties and the feature selection algorithm performance in terms of several criteria. We arrange the input meta-features into eight categories: (i) simple, (ii) statistical, (iii) information-theoretical, (iv) complexity, (v) landmarking, (vi) based on symbolic models, (vii) based on images, and (viii) based on complex networks (graphs). The target meta-features emerge from a multi-criteria performance measure, based on five individual performance indexes, that assesses feature selection methods grounded in information, distance, dependence, consistency, and precision measures. We evaluate our proposal using a recently developed framework that extracts the input meta-features from 213 benchmark datasets, and ranks the assessed feature selection algorithms, to fill in the target meta-features in meta-bases. This evaluation uses five state-of-the-art classification methods to induce recommendation models from meta-bases: C4.5, Random Forest, XGBoost, ANN, and SVM. The results showed that it is possible to reach an average accuracy of up to 90% applying our meta-feature engineering model. This work is the first to use an extensive empirical evaluation to provide a careful discussion of the strengths and limitations of more than 160 meta-features. These meta-features, while designed to aid the task of feature selection algorithm recommendation, can readily be employed in other metalearning scenarios. Therefore, we believe our findings are a valuable contribution to the fields of automated machine learning and data mining, as well as to the feature extraction and pattern recognition communities. • A novel meta-feature engineering model recommends feature selection algorithms. • The proposal obtains promising results from 213 datasets with hit rates of up to 90%. • Some simple, landmarking, image, and graph-based input meta-features highlighted. • A multi-criteria performance measure rigorously assesses candidate algorithms. • Chains of binary or multiclass classifiers can efficiently rank candidate algorithms. [ABSTRACT FROM AUTHOR]

• A machine learning network traffic analysis system has been designed. • Experiments of our proposed machine learning classifiers on a real-world P2P network traffic classification problem. • P2P network traffic classification using various machine learning techniques. • Comparison Performance of all Ensemble and Single models was performed using VPN netoworks. Network traffic classification is a significant and problematic aspect of network resource management arising from an investigation of network developments, planning, and design for 5G and beyond. Recently, traffic investigation systems for network monitoring and user access restrictions to Virtual Private Networks (VPN) and non-Virtual Private Networks (non-VPN) have gained widespread attention. In this paper, different algorithms for classifying and detecting VPN traffic are considered. A few existing machine learning procedures were tested concerning their performance in network traffic classification and security. The purpose is to improve Precision, Recall, and F1-score in VPN Network Traffic using Ensemble Classifiers. Therefore, the parameters of the ensemble classifier were changed to obtain high Precision, Recall, and F1-score. Bagging Decision Tree and Gradient Boosting algorithms were used for classification which produced promising results when compared to single classifiers like k -Nearest Neighbors (kNN), Multilayer Perceptron (MLP), and Decision Tree. The proposed classifier demonstrates recognition accuracy on a test sample of up to 93.80 % which outperforms all other single algorithms used in previous work. The MLP, Random Forest (RF), and Gradient Boosting (GB) algorithms had almost identical performance in all experiments. Furthermore, the proposed classifiers are found to perform better when the network traffic flows are generated using different values of time parameters (timeout). Our results show that the ensemble algorithms (Random Forest and the Gradient Boosting) outperform the single machine learning classifier previously used by other researchers, and we achieved the highest accuracy with the random forest classifier with better results while using non-VPN traffic and VPN traffic. The novelty lies in the application of an ensemble algorithm to secure a network traffic classification performed in comparison with single classifiers to determine Accuracy, Precision, and F1-score of a given dataset, contrary to the known process of selection of features and generation. [ABSTRACT FROM AUTHOR]