Search

In mechanical field, tool state seriously affects the production quality and efficiency. Recently, machine learning is an important technique to realize tool state recognition. Extracting and selecting features from the raw input data of the machine learning model can effectively eliminate useless information and improve the model accuracy. Therefore, a two-stage feature selection method is proposed in this paper. We first qualitatively propose the non-discreteness and separability of signal features. For these two properties, quantitative calculation methods are then developed in two stages. The proposed two-stage feature selection method is applied to machine learning models for tool state recognition. In engineering application verification, the proposed method achieves an average recognition accuracy of 84.4% in hob wear recognition, and an average recognition accuracy of 99.8% and 94.4% for two data sets in hob fault recognition, which is significantly higher than other feature selection methods. The verification results indicate that the proposed method has good generalization and engineering application value. [ABSTRACT FROM AUTHOR]

Developing a high-performing traffic incident-duration prediction model is considered a key component for evaluating the impact of these incidents on the roadway network. Various research studies have developed robust incident-duration prediction models. Still, they have faced many issues in providing an accurate prediction result due to the countless data modeling issues, such as complex correlations, highly skewed data distributions, heteroscedasticity, and outliers. This paper investigates the impact of feature optimization (FO) - a relatively new term encompassing two already-known topics: feature engineering (FE) and feature selection (FS) techniques - on the performance of several machine learning models developed for predicting incident durations. The models developed included multivariate linear regression, decision trees, support vector regressors, K-Nearest Neighbors regression, ensembles, and artificial neural networks. Various FO techniques have been used for each model to derive the massive traffic incidents dataset and repeat the prediction process. Our results show that the proposed filtering, wrapper, and embedded FS techniques can successfully reduce the number of features without sacrificing the prediction performance. Using log-normal transformation to deal with continuous data skewness, min-max normalization to deal with data variability, and principal component analysis (PCA) to reform the dataset into a smaller independent feature subset, FE techniques can enhance the accuracy of incident duration estimation over the assessed ML models. The best-performing FE technique was the PCA since performance improvements were observed across all developed ML models. The best-performing FS technique was the Recursive Feature Elimination, outperforming other tested techniques in reducing model complexity while maintaining model accuracy. [ABSTRACT FROM AUTHOR]

The increased penetration of renewable energy sources (RESs) as an effective mechanism to reduce carbon emissions leads to an increased weather dependency for power and energy systems. This has created dynamic operation and degradation phenomena, which affect the lifetime estimation of the assets operated with RESs. For the reliable and efficient operation of RES it is crucial to monitor the health of its constituent components and feature selection is a crucial step for building robust and accurate health monitoring approaches. In this context, this paper presents a probabilistic feature selection approach, which probabilistically weights and selects features through a heuristic and iterative process for an improved asset lifetime estimation. Power transformers are key power grid assets and they are used to demonstrate the validity and impact of the proposed approach. The approach is tested on two different photovoltaic power plants operated in Spain and Australia. Results consistently show that the proposed feature-selection approach reduces the prediction error and consistently selects relevant features. The approach has been applied to transformer lifetime estimation, but it can be generally applied to assist in the lifetime estimation of other components operated in RESs. Part of the studies presented here as well as source codes are all open-source under the GitHub repository https://github.com/iramirezg/FeatureSelection. • Probabilistic feature selection approach for improved asset lifetime estimation. • Integration of environmental features for improved renewable-operated asset lifetime. • Systematic and robust feature weighting methodology. • Improved transformer lifetime estimation including sensor and environmental data. • Validated on two real photovoltaic power plant case studies. [ABSTRACT FROM AUTHOR]

Conventional malware detection approaches have the overhead of feature extraction, the requirement of domain experts, and are time-consuming and resource-intensive. Learning-based approaches are the mainstay of malware detection as they overcome most of these challenges by significantly improving the detection effectiveness and providing a low false positive rate. The exponential growth of malware variants and first-time-appeared malware, which includes polymorphic and zero-day attacks, are some of the significant challenges to learning-based malware detectors. These challenges have catastrophic impacts on the detection effectiveness of these learning-based malware detectors. This paper proposes a novel deep learning-based framework to detect first-time-appeared malware effectively and efficiently by providing better performance than conventional malware detection approaches. First, it translates and visualises each Windows portable executable (PE) file into a coloured image to eliminate the overhead of feature extraction and the need for domain experts to analyse the features. In the subsequent step, a fine-tuned deep learning model is used to extract the deep features from the last fully connected layer. The step has reduced the cost of training required by the deep learning models if used for end-to-end classification. The third step selects the most important and influential features through a powerful feature selection algorithm. The most important features are then fed to a one-class classifier for final detection. With the one-class classifier, an enclosed boundary around the features of benign data is constructed. Anything outside the boundary is declared as an anomaly/malicious. It has enhanced the framework's ability to detect evolving, unseen, polymorphic, and zero-day attacks, as well as reducing the problem of overfitting. The detection effectiveness of the proposed framework is validated with state-of-the-art deep learning models and conventional approaches. The proposed framework has outperformed with an accuracy of 99.30% on the Malimg dataset. The Wilcoxon signed-rank test is used to validate the statistical significance of the proposed framework. It is evident from the results that the proposed framework is effective and can be used in the defence industry, resulting in more powerful and robust solutions against zero-day and polymorphic attacks. [Display omitted] • A novel approach of combining deep learning and machine learning is proposed. First, deep learning is used to extract deep features. The most influential and meticulous features are selected in the subsequent steps to train the machine learning classifier for final detection. The proposed framework eliminates the need for human efforts for reverse engineering tasks. • The proposed framework consists of four steps. In the first step, all PEs are transformed into coloured images. The second step used a deep learning model to extract the deep features. The subsequent step selects the most important features. Finally, the lightweight and most influential features are sent to the final machine learning classifier for final malware detection. • We demonstrate that the proposed framework is lightweight, resilient, efficient and cost-effective. An in-depth analysis is performed to validate the detection effectiveness and generalisation of the proposed framework on multiple datasets. Our results demonstrate that the proposed framework outperformed conventional and state-of-the-art malware detection approaches. [ABSTRACT FROM AUTHOR]

Attribute reduction can remove data noise and redundancy, thus reducing computational complexity, which is very important for machine learning. Because the difference between nominal attribute values is difficult to measure, attribute reduction for hybrid data faces challenges. In addition, most of the existing methods are sensitive to noise due to the lack of an anti-noise mechanism. Decision attribute contains the most important information of data. This paper proposes some techniques that consider the above problems from the perspective of fuzzy evidence theory. First of all, a new distance incorporating decision attributes is defined, and then a new fuzzy relation with an anti-noise mechanism is defined. Furthermore, fuzzy belief and fuzzy plausibility are defined based on the defined new distance and new fuzzy relation. In this framework, two anti-noise attribute reduction algorithms for hybrid data are proposed. Experiments on 12 data sets of various types show that compared with the other 8 state-of-the-art algorithms, the proposed algorithms improve the classification accuracy by at least 2% and the anti-noise ability by at least 11%. Therefore, it can be concluded that the proposed algorithms have excellent anti-noise ability while maintaining good feature selection ability. [ABSTRACT FROM AUTHOR]

Abstract This paper proposes a multitarget prediction approach using a new type of sphere complex fuzzy sets. The proposed model can adjust the number of outputs depending on the application. Moreover, a corresponding feature selection algorithm based on influence information and selection gain is proposed to solve the problem of cross-target feature selection in multitarget prediction. The proposed approach is tested using three experiments. The experimental results reveal that the model can predict at least four targets simultaneously and that the proposed feature selection algorithm can, in most cases, quickly determine a favorable feature combination. Highlights • This paper proposes novel sphere complex fuzzy sets (SCFSs), in contrast to conventional fuzzy sets, to improve modeling flexibility on the number of outputs, without significantly increasing parameters. • A modified PSO–RLSE hybrid learning algorithm is used to optimize the parameters of the multitarget prediction model based on the strategy of divide-and-conquer. • A new feature selection algorithm based on influence information and selection gain is presented for the cross-target feature selection problem derived from multi-target prediction. [ABSTRACT FROM AUTHOR]

This paper proposes a data-driven approach to estimating geometric track irregularities from instrumented railway vehicle (IRV) data. Machine learning is used to find the nonlinear mapping between IRV data and track irregularities. A dynamic model of the BRA1 railway vehicle was used to generate an artificial dataset that contains variables that are measured by the real BRA1 IRV and other variables measured by IRVs found in the literature. An extensive data analysis step was done to verify if the current instrumentation of the BRA1 IRV is sufficient for obtaining both lateral and vertical track irregularities. Feature engineering based on wagon movements, signal integration and time domain statistical metrics were applied to extract features and then the best features were selected using a wrapper method. Eight different regression ML models were trained and optimized after the feature selection using Optuna. The results show that, with the current instrumentation of the BRA1 IRV, obtaining lateral track irregularities is unlikely due to low correlation, however, vertical irregularities can be obtained with a root mean squared error (RMSE) of 0.556 mm. With postprocessing, the RMSE was further reduced to 0.410 mm. [ABSTRACT FROM AUTHOR]

The complex internal structure and heterogeneous composition material of aerospace equipment make it difficult to apply traditional acoustic emission source localization methods to the loose particle localization research. In previous studies, the authors transformed the loose particle localization problem into the multi-classification problem in machine learning, trained the loose particle localization model, but there were problems of low classification accuracy and low practicality. In this paper, the authors first introduced the ensemble learning idea into the loose particle detection field, designed a complete loose particle localization scheme, analyzed the above problems from multiple perspectives, and proposed new algorithms or strategies to enhance the superiority and practicability of the loose particle localization model. Specifically, in view of the low classification accuracy, the authors carried out the research on pulse preprocessing, feature engineering and model training from the four perspectives of signal, feature, data set and classifier, respectively. The zero-pulse-filling pulse matching algorithm and the channel-weighting-based feature selection method was newly proposed, Mel-Frequency Cepstral Coefficients features was newly extracted, feature optimization scheme was designed, and XGBoost ensemble classifier was trained. Thus, high-quality loose particle signals, high-quality localization data sets and high-performance loose particle localization models were obtained, respectively. Test results show that, the classification accuracy achieved by the new loose particle localization model was 96.80%, which was a significant improvement compared to the 83.53% achieved in previous studies. In view of the low practicality, the authors built the loose particle localization experimental system, gave the specific implementation steps of the loose particle localization method, as well as the general procedures for applying the loose particle localization model for physical testing. Meanwhile, taking into account the requirements of aerospace engineering applications, the authors added the majority voting strategy to convert classification results into localization results, thus newly proposed the definition of equipment-level loose particle localization accuracy. Several physical testing results show that, the loose particle localization model achieved a localization accuracy of 90.91%, which effectively verified the feasibility and stability of the proposed method. This study is an important supplement to the loose particle detection research, and of great significance in improving the reliability of aerospace systems. [ABSTRACT FROM AUTHOR]

Restricted by factors such as small robot size and harsh operating environment, the inability to obtain the interaction force between the robot arm and the environment through force sensors has become problematic in promoting the application of teleoperation systems in minimally invasive surgery, nuclear waste cleanup, and other fields. To accurately obtain the interaction force without the force sensors, a force observer based on an adaptive sparse general regression neural network (ASGRNN) is proposed in this paper. The proposed force observer uses a machine learning-based approach to obtain estimated force, thus eliminating the need for the dynamic parameters of the robot arm. Also, an innovative feature selection method incorporating the wrapper method and sparse regularization is proposed to select the input features of the force observer. Secondly, two new criteria are defined to eliminate the useless support vectors in the model. In addition, an improved antlion optimization algorithm (IALO) is proposed to optimize the bandwidth parameters of the model. To verify the performance of the proposed force observer, a 6-degree-of-freedom teleoperation robot experimental platform is built and compared with three existing force estimation models. The results show that the proposed force observer outperforms the existing model in terms of estimation accuracy, and the mean square error (MSE) is at least 35.79% lower than the existing model. In conclusion, this paper provides a feasible and effective force observer for a teleoperation system where force sensors are not applicable and dynamic parameters are non-knowable. • A new interaction force observer based on machine learning is proposed. • The proposed force observer does not require a force sensor and does not rely on any robot dynamics parameters. • A new feature selection method is proposed for selecting the input features of the force observer. • An improved ant lion optimization algorithm is proposed to optimize the force observer parameters. • The proposed force observer can accurately predict the interaction forces between the robot arm and the environment. [ABSTRACT FROM AUTHOR]

Locating low-velocity impacts (LVIs) on composite plates precisely is necessary. Support vector regression (SVR) is an effective method in addressing the LVI localization problem, whereas a large number of impact features may lead to a slow computational rate and over-fitting of the SVR model. In this paper, a binary dynamic stochastic search algorithm (BDSS) is proposed by introducing a threshold factor and the encoding concept of genetic algorithm into the original dynamic stochastic search algorithm (DSS). Then, a feature selection method is proposed by combining BDSS with SVR, which is called BDSS-SVR. BDSS-SVR as a wrapper method can simultaneously implement dimensionality reduction of impact features and optimize the SVR model's parameters. Furthermore, a novel LVI localization method based on BDSS-SVR and multi-domain features is designed for accurately detecting the LVIs on a carbon fiber reinforced plastic (CFRP) plate. To analyze the performance of BDSS-SVR, a LVI localization system based on fiber Bragg grating (FBG) sensors is applied to the conduction of two experiments. Two additional control parameters of BDSS-SVR, including the threshold factor and weight coefficient, are tuned in the first experiment. Moreover, the statistical results in the second experiment illustrate that BDSS-SVR outperforms optimized SVR using DSS, feature selection methods based on five state-of-the-art algorithms and SVR, and five machine learning methods. For fifteen random LVIs on the CFRP plate, BDSS-SVR effectively reduces the number of impact features and provides satisfactory localization accuracy. The maximum, minimum, and average errors are 4.659 mm, 0.024 mm, and 3.065 mm, respectively. [ABSTRACT FROM AUTHOR]

Feature selection aims to select crucial features to improve classification accuracy in machine learning and data mining. Existing methods concentrate on the classification information from candidate features while seldom considering the changing information supported by selected features. In this paper, we construct a dynamic support ratio (DSR), which employs the new information of selected features to support classification. DSR explicitly describes the dynamic interactions between selected features and candidate features. Simultaneously, the feature relevance and feature redundancy are treated adaptively. Thus, distinctive features can be noticed sensitively. Afterward, a novel feature selection method based on a dynamic support ratio (DSRFS) is proposed. The proposed method is established on 18 benchmark data sets with four different classifiers. Classification accuracy, standard deviation, recall and statistical validations are employed to measure the classification performance. Extensive experiments demonstrate that DSRFS not only reduces the dimension of the feature space effectively, but also obtains the best average classification accuracy. [ABSTRACT FROM AUTHOR]

Cancer is considered to be a key cause of substantial fatality and morbidity in the world. A report from the International Agency for Research on Cancer (IARC) states that 27 million new cases of cancer are expected before 2030. 1 in 18 men and 1 in 46 women are estimated to develop lung cancer over a lifetime. This paper discusses an overview of lung cancer, along with publicly available benchmark data sets for research purposes. Recent research performed in medical image analysis of lung cancer using deep learning algorithms is compared using various technical aspects such as efficiency, advantages, and limitations. These discussed approaches provide insight into techniques that can be used to perform the detection and classification of lung cancer. Numerous techniques adapted in the acquisition of the images, extraction of relevant features, segmentation of region affected, selection of optimal features, and classification are also discussed. The paper is concluded by stating the clinical, technical challenges and prominent future directions. [ABSTRACT FROM AUTHOR]

One of the most important tasks in pattern, machine learning, and data mining is classification problem. Introducing a general classifier is a challenge for pattern recognition communities, which enables one to learn each problem׳s dataset. Many classifiers have been proposed to learn any problem thus far. However, many of them have their own positive and negative aspects. So they are good only for specific problems. But there is no strong solution to recognize which classifier is better or good for a specific problem. Fortunately, ensemble learning provides a good way to have a near-optimal classifying system for any problem. One of the most challenging problems in classifier ensemble is introducing a suitable ensemble of base classifiers. Every ensemble needs diversity. It means that if a group of classifiers is to be a successful ensemble, they must be diverse enough to cover their errors. Therefore, during ensemble creation, a mechanism is needed to ensure that the ensemble classifiers are diverse. Sometimes this mechanism can select/remove a subset of base classifiers with respect to maintaining the diversity of the ensemble. This paper proposes a novel method, named the Classifier Selection Based on Clustering (CSBS), for ensemble creation. To insure diversity in ensemble classifiers, this method uses the clustering of classifiers technique. Bagging is used to produce base classifiers. During ensemble creation, every type of base classifier is the same as a decision tree classier or a multilayer perceptron classifier. After producing a number of base classifiers, CSBC partitions them by using a clustering algorithm. Then CSBC produces a final ensemble by selecting one classifier from each cluster. Weighted majority vote method is used as an aggregator function. In this paper we investigate the influence of cluster number on the performance of the CSBC method; we also probe how we can select a good approximate value for cluster number in any dataset. We base our study on a large number of real datasets of UCI repository to reach a definite result. [ABSTRACT FROM AUTHOR]

Knowledge representation and the ability to learn new knowledge define the success of situation recognition within cognitive systems. In this contribution, it is assumed that a complex environment in which a system is acting is modeled using an event-discrete approach. The modeling would be based on imprecise and uncertain knowledge about the environment, which is adapted by suitable learning abilities during the interaction between the system and environment. The main contribution of this paper is developing an approximate reasoning approach driven by learning and reusing human-operator experiences to handle event-discrete situations in unknown dynamic environments. A new approach for modeling and learning the knowledge for situation recognition in human operator assistance systems is proposed. Situation recognition is individualized for humans by learning exclusive experiences of human operators in interaction with the environment. Individualization is caused by variety of human operators in definition of priorities and goals, generation of events, and different environmental characteristics importance in specification of situations. For modeling interaction-based knowledge structures, a fuzzy Situation-Operator Modeling approach is used and improved by applying a feature selection process. The corresponding approach is able to learn and represent new knowledge to improve the performance of individualized situation recognition for cognitive systems. Here, the proposed approach is applied using a simulated driving environment and evaluated for different test drivers. Evaluation results highlight the ability and importance of the proposed approach for situation recognition in driving assistance systems. [ABSTRACT FROM AUTHOR]

Gene selection (or feature selection) from DNA-microarray data can be focused on different techniques, which generally involve statistical tests, data mining and machine learning. In recent years there has been an increasing interest in using hybrid-technique sets to face the problem of meaningful gene selection; nevertheless, this issue remains a challenge. In an effort to address the situation, this paper proposes a novel hybrid framework based on data mining techniques and tuned to select gene subsets, which are meaningfully related to the target disease conducted in DNA-microarray experiments. For this purpose, the framework above deals with approaches such as statistical significance tests, cluster analysis, evolutionary computation, visual analytics and boundary points. The latter is the core technique of our proposal, allowing the framework to define two methods of gene selection. Another novelty of this work is the inclusion of the age of patients as an additional factor in our analysis, which can leading to gaining more insight into the disease. In fact, the results reached in this research have been very promising and have shown their biological validity. Hence, our proposal has resulted in a methodology that can be followed in the gene selection process from DNA-microarray data. [ABSTRACT FROM AUTHOR]

With increasing number of documents in digital format, automatic text categorization has become a crucial task in pattern recognition problems. To ease the classification task, feature selection methods have been introduced to reduce the dimensionality of the feature space, and thus improve the classification performance. In this paper a novel filter method for feature selection, called Multivariate Relative Discrimination Criterion (MRDC), is proposed for text classification. The proposed method focuses on the reduction of redundant features using minimal-redundancy and maximal-relevancy concepts. To this end, the proposed method takes into account document frequencies for each term, while estimating their usefulness. The proposed method not only selects the features with maximum relevancy, but also the redundancy between them is takes into account using a correlation metric. MRDC does not employ any learning algorithm to evaluate the usefulness of the selected features, and thus it can be categorized as a filter method. In order to assess the effectiveness of the proposed method, several experiments are performed on three real-world datasets. The obtained results are compared to the state-of-the-art filter methods. The reported results show that in most cases MRDC results in better classification performance than others. [ABSTRACT FROM AUTHOR]

The interest in solar radiation prediction has increased greatly in recent times among the scientific community. In this context, Machine Learning techniques have shown their ability to learn accurate prediction models. The aim of this paper is to go one step further and automatically achieve interpretability during the learning process by performing dimensionality reduction on the input variables. To this end, three non standard multivariate feature selection approaches are applied, based on the adaptation of strong learning algorithms to the feature selection task, as well as a battery of classic dimensionality reduction models. The goal is to obtain robust sets of features that not only improve prediction accuracy but also provide more interpretable and consistent results. Real data from the Weather Research and Forecasting model, which produces a very large number of variables, is used as the input. As is to be expected, the results prove that dimensionality reduction in general is a useful tool for improving performance, as well as easing the interpretability of the results. In fact, the proposed non standard methods offer important accuracy improvements and one of them provides with an intuitive and reduced selection of features and mesoscale nodes (around 10% of the initial variables centered on three specific nodes). [ABSTRACT FROM AUTHOR]

Abstract: In food quality monitoring, color is an important indicator factor of quality. The CIELab (L⁎a⁎b⁎) color space as a device independent color space is an appropriate means in this case. The commonly used colorimeter instruments can neither measure the L⁎a⁎b color in a wide area over the target surface nor in a contact-less mode. However, developing algorithms for conversion of food items images into L⁎a⁎b color space can solve both of these issues. This paper addresses the problem of L⁎a⁎b color prediction from multispectral images of different types of raw meat. The efficiency of using multispectral images instead of the standard RGB is investigated. In addition, it is demonstrated that due to the fiber structure and transparency of raw meat, the prediction models built on the standard color patches do not work for raw meat test samples. As a result, multispectral images of different types of meat samples (430–970nm) were used for training and testing of the L⁎a⁎b prediction models. Finding a sparse solution or the use of a minimum number of bands is of particular interest to make an industrial vision set-up simpler and cost effective. In this paper, a wide range of linear, non-linear, kernel-based regression and sparse regression methods are compared. In order to improve the prediction results of these models, we propose a supervised feature selection strategy which is compared with the Principal component analysis (PCA) as a pre-processing step. The results showed that the proposed feature selection method outperforms the PCA for both linear and non-linear methods. The highest performance was obtained by linear ridge regression applied on the selected features from the proposed Elastic net (EN) -based feature selection strategy. All the best models use a reduced number of wavelengths for each of the L⁎a⁎b components. [Copyright &y& Elsevier]

Microarray high dimensional datasets suffer from small sample size and extreme large number of features. Therefore, feature selection plays crucial roles on the performance of the trained models on those datasets. A typical feature selection method consists of two main parts, problem criterion and a search strategy. The common datasets don’t have huge number of features with respect to their number of samples; hence, a search strategy in their feature selection methods were able to seek the search space. In contrast, microarray high dimensional datasets have huge number of features; therefore, their search space is very large and searching that space is a prohibitive action. In this paper, we take into account the philosophy of Occam's razor in feature subset selection in order to release high dimensional datasets from computational search methods. The proposed method uses two stages for feature selection. In the first stage features are rearranged by their importance in the dataset and in the second stage, the fundamental concept of reduced row Echelon form is applied on dataset in order to find linear independent features. For determining the effectiveness of the proposed method some experiments are carried out on nine binary microarray high dimensional datasets. The obtained results are compared with eleven state-of-the-art feature selection algorithms including Correlation based Feature Selection (CFS), Fast Correlation Based Filter (FCBF), Interact (INT) and Maximum Relevancy Minimum Redundancy (MRMR). The average outcomes of the results are analyzed by a statistical non-parametric test and it reveals that the proposed method has a meaningful superiority to the others in terms of accuracy, sensitivity, specificity, G-mean, number of selected features and computational complexity. [ABSTRACT FROM AUTHOR]

The progress of machine learning (ML) in the past years has opened up new opportunities to the field of gas turbine (GT) modelling. However, successful implementation of ML algorithms remains challenging, particularly for complex problems such as multi-mode faults. An important tool for enabling applications are the feature selection and feature learning (FSFL) techniques. In particular, FL techniques have recently facilitated and improved the applicability of ML to GT modelling. This review paper conducts a review on 46 studies that utilised FSFL for GT modelling with ML. The purpose of this review is to investigate how FSFL techniques can help address GT modelling challenges and when researchers should deploy them. Therefore, the theories behind the techniques are illustrated in depth along with practical application examples from the analysed literature. The advantages and limitations of FSFL are discussed, the computational costs of different techniques are compared, and trends in the field are highlighted. Consequently, a novel categorisation framework for FSFL techniques and recommendations regarding when and how to implement them are provided. A new knowledge accumulation, extraction, and transfer concept is proposed to address GT modelling challenges. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a core vector machine (CVM)-based algorithm for on-line voltage security assessment of power systems. To classify the system security status, a CVM has been trained for each contingency. The proposed CVM-based security assessment has very small training time and space in comparison with support vector machines (SVM) and artificial neural networks (ANNs)-based algorithms. The proposed algorithm produces less support vectors (SV) and therefore is faster than existing algorithms. In this paper, a new decision tree (DT)-based feature selection technique has been presented, too. The proposed CVM algorithm has been applied to New England 39-bus power system. The simulation results show the effectiveness and the stability of the proposed method for on-line voltage security assessment procedure of large-scale power system. [Copyright &y& Elsevier]

Feature selection, which aims to screen out redundant and irrelevant features from datasets, is integral to machine learning and data mining. Grey Wolf Optimization (GWO) is a recent meta-heuristic algorithm based on swarm intelligence and has wide applicability to various optimization problems due to its fast convergence and few parameters. However, since the wolf pack is always dominated by the three leading wolves (i.e. , α , β and δ), the GWO algorithm suffers from weak exploration throughout the whole optimization process and easily stagnates into local optima. In this paper, an Adaptively Balanced Grey Wolf Optimization (ABGWO) algorithm is proposed to seek out the optimal feature subset for high-dimensional classification. Specifically, to improve the exploration ability of GWO, a random wolf is introduced to cooperate with α , β and δ. A novel level-based strategy is further adopted to select the random wolf. Besides, to dynamically modulate the exploration and exploitation ability in different optimization stages, an adaptive coefficient is introduced to regulate the leadership of α , β , δ and the randomly-selected wolf. Finally, the improvement of exploration and exploitation is validated on 12 high-dimensional datasets provided by Arizona State University and University of California Irvine, and the superiority of ABGWO is further verified by comparing it with seven state-of-the-art feature selection approaches on the aspect of classification accuracy, size of feature subset and computational time. • Introducing a random individual to prevent sharp decline in population diversity. • Semi-random selection prevents the rapid decrease of the convergence speed. • A coefficient factor linearly increases to adaptively adjust the exploration and exploitation. • The improved GWO algorithm for high-dimensional feature selection significantly boosts the classification performance. [ABSTRACT FROM AUTHOR]

This paper proposes an ensemble algorithm named of BPSO-Adaboost-KNN to cope with multi-class imbalanced data classification. The main idea of this algorithm is to integrate feature selection and boosting into ensemble. What’s more, we utilize a novel evaluation metric called AUCarea which is especially for multi-class classification. In our model BPSO is employed as the feature selection algorithm in which AUCarea is chosen as the fitness. For classification, we generate a boosting classifier in which KNN is selected as the basic classifier. In order to verify the effectiveness of our method, 19 benchmarks are used in our experiments. The results show that the proposed algorithm improves both the stability and the accuracy of boosting after carrying out feature selection, and the performance of our algorithm is comparable with other state-of-the-art algorithms. In statistical analyses, we apply Bland–Altman analysis to show the consistencies between AUCarea and other popular metrics like average G-mean, average F -value etc. Besides, we use linear regression to find deeper correlation between AUCarea and other metrics in order to show why AUCarea works well in this issue. We also put out a series of statistical studies in order to analyze if there exist significant improvements after feature selection and boosting are employed. At last, the proposed algorithm is applied in oil-bearing of reservoir recognition. The classification precision is up to 99% in oilsk81-oilsk85 well logging data in Jianghan oilfield of China, which is 20% higher than KNN classifier. Particularly, the proposed algorithm has significant superiority when distinguishing the oil layer from other layers. [ABSTRACT FROM AUTHOR]

In the past decades, the rapid growth of computer and database technologies has led to the rapid growth of large-scale datasets. On the other hand, data mining applications with high dimensional datasets that require high speed and accuracy are rapidly increasing. An important issue with these applications is the curse of dimensionality, where the number of features is much higher than the number of patterns. One of the dimensionality reduction approaches is feature selection that can increase the accuracy of the data mining task and reduce its computational complexity. The feature selection method aims at selecting a subset of features with the lowest inner similarity and highest relevancy to the target class. It reduces the dimensionality of the data by eliminating irrelevant, redundant, or noisy data. In this paper, a comparative analysis of different feature selection methods is presented, and a general categorization of these methods is performed. Moreover, in this paper, state-of-the-art swarm intelligence is studied, and the recent feature selection methods based on these algorithms are reviewed. Furthermore, the strengths and weaknesses of the different studied swarm intelligence-based feature selection methods are evaluated. • In this paper an overview of feature selection problem is provided. • Previous feature selection methods are categorized and the advantages and disadvantages of these methods are described. • Previous Swarm Intelligence-based feature selection methods are reviewed. • Experimental results of different Swarm Intelligence-based feature selection methods are reported. • The strengths and weaknesses of the previous Swarm Intelligence-based feature selection are discussed. [ABSTRACT FROM AUTHOR]

This paper copes with the COGERSA waste collection process. Up to now, experts have been manually designed the process using a trial and error mechanism. This process is not globally optimized, since it has been progressively and locally built as council demands appear. Planning optimization algorithms usually solve it, but they need a fitness function to evaluate a route planning quality. The drawback is that even experts are not able to propose one in a straightforward way due to the complexity of the process. Hence, the goal of this paper is to build a fitness function though a preference framework, taking advantage of the available expert knowledge and expertise. Several key performance indicators together with preference judgments are carefully established according to the experts for learning a promising fitness function. Particularly, the additivity property of them makes the task be much more affordable, since it allows to work with routes rather than with route plannings. Besides, a feature selection analysis is performed over such indicators, since the experts suspect of a potential existing (but unknown) redundancy among them. The experiment results confirm this hypothesis, since the best C − index (98% against around 94%) is reached when 6 or 8 out of 21 indicators are taken. Particularly, truck load seems to be a highly promising key performance indicator, together to the travelled distance along non-main roads. A comparison with other existing approaches shows that the proposed method clearly outperforms them, since the C − index goes from 72% or 90% to 98%. • A fitness function to evaluate a waste collection route planning. • Key performance indicators for a waste collection route planning. • Feature selection of key performance indicators for a waste collection route planning. [ABSTRACT FROM AUTHOR]

Restricted Boltzmann machines and deep belief networks have been shown to perform effectively in many applications such as supervised and unsupervised learning, dimensionality reduction and feature learning. Implementing networks, which use contrastive divergence as the learning algorithm on neuromorphic hardware, can be beneficial for real-time hardware interfacing, power efficient hardware and scalability. Neuromorphic hardware which uses memristors as synapses is one of the most promising areas to achieve the above-mentioned goals. This paper presents a restricted Boltzmann machine which uses a two memristor model to emulate synaptic weights and achieves learning using contrastive divergence. [ABSTRACT FROM AUTHOR]

Machine Learning techniques (ML) are being widely adopted in embedded devices due to their efficiency and flexibility. However, the strict power limitations in such devices, combined with the variable resource requirements of ML models, require further understanding of how model complexity affects power and performance. This paper proposes a framework that facilitates the design space exploration of dedicated decision trees (DT) and random forests (RF) accelerators by enabling a joint assessment of power dissipation and prediction accuracy. The proposed framework translates tree-based structures to hardware description languages (HDL). The HDL modules are submitted to logic and physically-aware hardware synthesis flows, allowing a detailed power-performance analysis of VLSI DTs and RFs. Using four data sets of embedded applications as case studies, we found that quantizing the input features leads to accuracy gains of up to 6.3% compared with the precise versions. We also show that using shallower trees may lead to small prediction loss with significant reductions in power, which is favorable for power-constrained applications. Our translator achieves better results in terms of energy/inference w.r.t. prior related works under comparison, one of which employed standard methods for hardware translation such as High-Level Synthesis. The proposed solution presents a power reduction of 10 times or more for the same inference throughput reported in prior works. [ABSTRACT FROM AUTHOR]

In this paper we propose three novel feature ranking methods for supervised feature selection in the context of classification which are based on possibility theory. All three methods – nonspecificity, strife and total uncertainty – are tested on eight artificial data sets and ten medical real-world data sets and benchmarked against ReliefF, the Fisher score, the fuzzy entropy and similarity (FES), the Fuzzy similarity and entropy (FSAE) filter, symmetrical uncertainty as well as using no feature selection. The feature ranking methods were applied following two approaches: (1) using a fixed threshold for the number of highest-ranking features selected and (2) using a hybrid feature selection approach with a classifier (k-nearest neighbor classifier, decision tree, similarity classifier, SVM) to select the optimal number of features to select. The results indicate that strife and the Fisher score are the two feature ranking methods that for both approaches are on average ranked the highest in terms of the test set accuracy on the real-world data sets. Besides that, for the hybrid approach, strife uses most of the time a considerably smaller number of features than nonspecificity and total uncertainty. In terms of stability, which was measured with the adjusted stability measure (ASM), the Fisher score and strife were among the most stable feature ranking methods in this study. Additionally, strife's feature subsets were diverse compared to those of the remaining feature selection methods, making it a good candidate to be included in a feature selection ensemble. [ABSTRACT FROM AUTHOR]

Abstract: Feature selection is a combinatorial optimization problem that selects most relevant features from an original feature set to increase the performance of classification or clustering algorithms. Most feature selection methods are supervised methods and use the class labels as a guide. On the other hand, unsupervised feature selection is a more difficult problem due to the unavailability of class labels. In this paper, we present an unsupervised feature selection method based on ant colony optimization, called UFSACO. The method seeks to find the optimal feature subset through several iterations without using any learning algorithms. Moreover, the feature relevance will be computed based on the similarity between features, which leads to the minimization of the redundancy. Therefore, it can be classified as a filter-based multivariate method. The proposed method has a low computational complexity, thus it can be applied for high dimensional datasets. We compare the performance of UFSACO to 11 well-known univariate and multivariate feature selection methods using different classifiers (support vector machine, decision tree, and naïve Bayes). The experimental results on several frequently used datasets show the efficiency and effectiveness of the UFSACO method as well as improvements over previous related methods. [Copyright &y& Elsevier]

Abstract: Traditional cross-domain learning methods transfer learning from a source domain to a target domain. In this paper, we propose the multiple-domain learning problem for several equally treated domains. The multiple-domain learning problem assumes that samples from different domains have different distributions, but share the same feature and class label spaces. Each domain could be a target domain, while also be a source domain for other domains. A novel multiple-domain representation method is proposed for the multiple-domain learning problem. This method is based on nonnegative matrix factorization (NMF), and tries to learn a basis matrix and coding vectors for samples, so that the domain distribution mismatch among different domains will be reduced under an extended variation of the maximum mean discrepancy (MMD) criterion. The novel algorithm — multiple-domain NMF (MDNMF) — was evaluated on two challenging multiple-domain learning problems — multiple user spam email detection and multiple-domain glioma diagnosis. The effectiveness of the proposed algorithm is experimentally verified. [Copyright &y& Elsevier]

Abstract: Since given classification data often contains redundant, useless or misleading features, feature selection is an important pre-processing step for solving classification problems. This problem is often solved by applying evolutionary algorithms to decrease the dimensional number of features involved. Removing irrelevant features in the feature space and identifying relevant features correctly is the primary objective, which can increase classification accuracy. In this paper, a novel QBGSA–K-NN hybrid system which hybridizes the quantum-inspired binary gravitational search algorithm (QBGSA) with the K-nearest neighbor (K-NN) method with leave-one-out cross-validation (LOOCV) is proposed. The main aim of this system is to improve classification accuracy with an appropriate feature subset in binary problems. We evaluate the proposed hybrid system on several UCI machine learning benchmark examples. The experimental results show that the proposed method is able to select the discriminating input features correctly and achieve high classification accuracy which is comparable to or better than well-known similar classifier systems. [Copyright &y& Elsevier]

Sine Cosine Algorithm (SCA) that is one of the population-based metaheuristic optimization algorithms basically consists of the updating mechanism based on sine and cosine functions. In this algorithm, a few random and adaptive variables are also utilized for more effective motions of the candidate solutions. SCA has some drawbacks like other some metaheuristic algorithms. SCA tends to be stuck into the local regions in the search space and this affects negatively on the computational effort required to find the best solution point in the search space. This paper presents four different improved versions of SCA. The proposed improvements on original SCA are the innovations on the updating mechanism of SCA. To evaluate the performances of Improved Sine Cosine Algorithms (ImpSCAs), well-known numerical optimization problems including CEC 2014 test suite are used. Firstly, different analyses of the proposed ImpSCAs are dealt with such as the convergence analysis, search history analysis, trajectory analysis, average distance analysis, and computational complexity analysis. Secondly, the proposed four versions of ImpSCAs are compared with the original SCA for CEC 2014 benchmark problems with dimension sizes of 10D, 30D and 50D. Finally, original SCA and ImpSCAs are adapted to select optimal feature combination and they are tested for 10 feature selection datasets taken from the UCI machine learning repository. The benchmark results show that the performances of the ImpSCA 1 , ImpSCA 2 , and ImpSCA 4 are better than that of the original SCA. From the feature selection results, it is observed that three versions of ImpSCAs (except ImpSCA 3) outperform the original SCA in 80% of the datasets. Source codes of ImpSCAs are publicly available at https://github.com/uguryuzgec/ImpSCAs. • We focused to improve the exploration and exploitation ability of the original SCA. • The basic idea is based on the innovations on the update mechanism. • The ISCAs were evaluated using CEC2014 benchmark test suite. • The proposed ISCAs were tested for 10 popular UCI datasets. • Three versions of ISCAs (except ISCA 3) outperform the SCA in 80% of the datasets. [ABSTRACT FROM AUTHOR]

Abstract: Motivated by the high demand to construct compact and accurate statistical models that are automatically adjustable to dynamic changes, in this paper, we propose an online probabilistic framework for high-dimensional spherical data modeling. The proposed framework allows simultaneous clustering and feature selection in online settings using finite mixtures of von Mises distributions (movM). The unsupervised learning of the resulting model is approached using Expectation Maximization (EM) for parameter estimation along with minimum message length (MML) to determine the optimal number of mixture components. The gradient stochastic descent approach is considered for incremental updating of model parameters, also. Through empirical experiments, we demonstrate the merits of the proposed learning framework on diverse high dimensional datasets and challenging applications. [Copyright &y& Elsevier]

Abstract: In this paper, we develop a diagnosis model based on particle swarm optimization (PSO), support vector machines (SVMs) and association rules (ARs) to diagnose erythemato-squamous diseases. The proposed model consists of two stages: first, AR is used to select the optimal feature subset from the original feature set; then a PSO based approach for parameter determination of SVM is developed to find the best parameters of kernel function (based on the fact that kernel parameter setting in the SVM training procedure significantly influences the classification accuracy, and PSO is a promising tool for global searching). Experimental results show that the proposed AR_PSO–SVM model achieves 98.91% classification accuracy using 24 features of the erythemato-squamous diseases dataset taken from UCI (University of California at Irvine) machine learning database. Therefore, we can conclude that our proposed method is very promising compared to the previously reported results. [Copyright &y& Elsevier]

The increasing popularity of the Internet led to a substantial growth of e-commerce. However, such activities have main security challenges primary caused by cyberfraud and identity theft. Therefore, checking the legitimacy of visited web pages is a crucial task to secure costumers' identities and prevent phishing attacks. The use of machine learning is widely recognized as a promising solution. The literature is rich with studies that use machine learning techniques for website phishing detection. However, their findings are dataset dependent and are far away from generalization. Two main reasons for this unfortunate state are the impracticable replication and absence of appropriate benchmark datasets for fair evaluation of systems. Moreover, phishing tactics are continuously evolving and proposed systems are not following those rapid changes. In this paper, we present a general scheme for building reproducible and extensible datasets for website phishing detection. The aim is to (1) enable comparison of systems adopting different features, (2) overtake the short-lived nature of phishing websites, and (3) keep track of the evolution of phishing tactics. For experimenting the proposed scheme, we start by adopting a refined categorization of website phishing features and we systematically select a total of 87 commonly recognized ones, we categorize them, and we made them subjects for relevance and runtime analysis. We use the collected set of features to build a dataset in light of the proposed scheme. Thereafter, we use a conceptual replication approach to check the genericity of former findings for the built dataset. Specifically, we evaluate the performance of classifiers on individual and combined categories of features, we investigate different combinations of models, and we explore the effects of filter and wrapper methods on the selection of discriminative features. The results show that Random Forest is the most predictive classifier. Features gathered from external services are the most discriminative where features extracted from web page contents are less distinguishing. Besides external service based features, some web page content features are found not suitable for runtime detection. The use of hybrid features provided the best accuracy score of 96.61%. By investigating different feature selection methods, filter-based ranking with incremental removal of less important features improved the performance up to 96.83% better than wrapper methods. [ABSTRACT FROM AUTHOR]

Abstract: The explosion of on-line information has given rise to many manually constructed topic hierarchies (such as Yahoo!!). But with the current growth rate in the amount of information, manual classification in topic hierarchies results in an immense information bottleneck. Therefore, developing an automatic classifier is an urgent need. However, classifiers suffer from enormous dimensionality, since the dimensionality is determined by the number of distinct keywords in a document corpus. More seriously, most classifiers are either working slowly or they are constructed subjectively without any learning ability. In this paper, we address these problems with a fair feature-subset selection (FFSS) algorithm and an adaptive fuzzy learning network (AFLN) for classification. The FFSS algorithm is used to reduce the enormous dimensionality. It not only gives fair treatment to each category but also has ability to identify useful features, including both positive and negative features. On the other hand, the AFLN provides extremely fast learning ability to model the uncertain behavior for classification so as to correct the fuzzy matrix automatically. Experimental results show that both FFSS algorithm and the AFLN lead to a significant improvement in document classification, compared to alternative approaches. [Copyright &y& Elsevier]

Machine Learning (ML) techniques are becoming an invaluable support for network intrusion detection, especially in revealing anomalous flows, which often hide cyber-threats. Typically, ML algorithms are exploited to classify/recognize data traffic on the basis of statistical features such as inter-arrival times, packets length distribution, mean number of flows, etc. Dealing with the vast diversity and number of features that typically characterize data traffic is a hard problem. This results in the following issues: (i) the presence of so many features leads to lengthy training processes (particularly when features are highly correlated), while prediction accuracy does not proportionally improve; (ii) some of the features may introduce bias during the classification process, particularly those that have scarce relation with the data traffic to be classified. To this end, by reducing the feature space and retaining only the most significant features, Feature Selection (FS) becomes a crucial pre-processing step in network management and, specifically, for the purposes of network intrusion detection. In this review paper, we complement other surveys in multiple ways: (i) evaluating more recent datasets (updated w.r.t. obsolete KDD 99) by means of a designed-from-scratch Python-based procedure; (ii) providing a synopsis of most credited FS approaches in the field of intrusion detection, including Multi-Objective Evolutionary techniques; (iii) assessing various experimental analyses such as feature correlation, time complexity, and performance. Our comparisons offer useful guidelines to network/security managers who are considering the incorporation of ML concepts into network intrusion detection, where trade-offs between performance and resource consumption are crucial. [ABSTRACT FROM AUTHOR]

In the context of natural language processing, accuracy of intention detection is the basis for subsequent research on human-machine speech interaction. However, the problem of ambiguity in word vectors reduces the accuracy of intent detection. Meantime, there is a disconnection between local features and global features as well, resulting in text feature extraction that cannot fully reflect semantic information. These issues are all barriers of intention detection. Therefore, this paper proposes an attention-based convolutional neural network for self-media data learning (called A-CNN) for marketing intention. We cascade the traditional CNN with the self-attention model in the Attention networks to form a new network structure called A-CNN, and put forward a fast feature extraction method based on skip-gram-based learning called FSLText, to represent the high-dimension word vectors in the A-CNN. On the premise of maintaining the advantages of the CNN, A-CNN can not only solve the problem of local and global features disconnection caused by the CNN pooling layer, but also avoid the increase of algorithm complexity. The Self-Attention mechanism in the Attention model can effectively optimize the weight of local features of the information in global features, and retain local features that are more useful for intention detection. A fast feature extraction method which is based on Skip-gram can retain the semantic and word order information of the text. The method is beneficial to the marketing intention detection. According to the experiment, our A-CNN, compared with traditional machine learning methods, can improve 12.32% accuracy. Contrast to the dual-channel CNN, the accuracy rate is improved by 9.68%, and compared with the ATT-CNN, it is improved by 9.97%. On the F1 score, the A-CNN can improve the F1 score by about 9.37% in comparison with the traditional machine learning methods, the accuracy rate is increased by 9.68% compared with the dual-channel CNN, and 9.68% in contrast with ATT-CNN. It illustrates that our A-CNN can effectively address semantic and feature selection for marketing intention detection. [ABSTRACT FROM AUTHOR]

Feature selection methods have an important role on the readability of data and the reduction of complexity of learning algorithms. In recent years, a variety of efforts are investigated on feature selection problems based on unsupervised viewpoint due to the laborious labeling task on large datasets. In this paper, we propose a novel approach on unsupervised feature selection initiated from the subspace clustering to preserve the similarities by representation learning of low dimensional subspaces among the samples. A self-expressive model is employed to implicitly learn the cluster similarities in an adaptive manner. The proposed method not only maintains the sample similarities through subspace clustering, but it also considers the underlying structure of data based on a regularized regression model. In line with the convergence analysis of the proposed method, the experimental results on benchmark datasets demonstrate the effectiveness of our approach as compared with the state-of-the-art methods. • Propose a novel unsupervised feature selection method based on subspace clustering and adaptive similarity learning. • Introduce a self-expressive model to adaptively and implicitly learn the cluster similarities. • Employ a regularized regression model to obtain the sparse correlation among the features and clusters. • Introduce a unified objective function to consider the main factors in the proposed approach. • The results show that our proposed approach outperformed several state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Today's real-world data mostly involves incomplete, inconsistent, and/or irrelevant information that causes many drawbacks to transform it into an understandable format. In order to deal with such issues, data preprocessing is a proven discipline in data mining. One of the typical tasks in data preprocessing, feature selection aims to reduce the dimensionality in the data and thereby contributes to further processing. Feature selection is widely used to enhance the performance of a supervised learning algorithm (e.g., classification) but is rarely used in unsupervised tasks (e.g., clustering). This paper introduces a new multi-objective differential evolution approach in order to find relatively homogeneous clusters without the prior knowledge of cluster number using a smaller number of features from all available features in the data. To analyze the goodness of the introduced approach, several experiments are conducted on a various number of real-world and synthetic benchmarks using a variety of clustering approaches. From the analyzes through several different criteria, it is suggested that our method can significantly improve the clustering performance while reducing the dimensionality at the same time. [ABSTRACT FROM AUTHOR]