Search

• A feature selection method based on binary dandelion algorithm is proposed. • The algorithm applies a sine-cosine operator and a restart strategy. • Mutual information and quick bit mutation improve the performance of the algorithm. • The algorithm performs well on datasets with larger dimensions. Feature selection (FS) is an important data preprocessing technology for machine learning and data mining. Metaheuristic algorithm (MH) has been widely used in feature selection because of its powerful search function. This paper presents an improved Binary Dandelion Algorithm using Sine Cosine operator and Restart strategy (SCRBDA) for feature selection. First, the sine cosine operator is used in the radius formula of the core dandelions (CD), which significantly enhances the ability of algorithm development and exploration. Secondly, the algorithm uses a restart strategy to increase its ability to get rid of local optimum. Thirdly, mutual information is used to guide the generation of some dandelions, which pays more attention to the correlation between the selected features and categories. Finally, quick bit mutation is used as the mutation strategy to improve the diversity of the population. The SCRBDA proposed in this paper was tested on 18 datasets of different sizes from UCI machine learning database. The SCRBDA was compared with 8 other classical feature selection algorithms, and the performance of the proposed algorithm was evaluated through feature subset size, classification accuracy, fitness value, and F1-score. The experimental results show that SCRBDA achieves the best performance, which has stronger feature reduction ability and achieves better overall performance on most datasets. Especially on large-scale datasets, SCRBDA can obtain extremely smaller feature subsets while maintaining much higher classification accuracy, and satisfactory F1-score. [ABSTRACT FROM AUTHOR]

Abstract: Quite often, in order to derive meaningful insights, accounting researchers have to analyze large bodies of text. Usually, this is done manually by several human coders, which makes the process time consuming, expensive, and often neither replicable nor accurate. In an attempt to mitigate these problems, we perform a feasibility study investigating the applicability of computer-aided content analysis techniques onto the domain of accounting research. defines an algorithm’s reliability as its stability, reproducibility and accuracy. Since in computer-aided text classification, which is inherently objective and repeatable, the first two requirements, stability and reproducibility, are not an issue, this paper focuses exclusively on the third requirement, the algorithm’s accuracy. It is important to note that, although inaccurate classification results are completely worthless, it is surprising to see how few research papers actually mention the accuracy of the used classification methodology. After a survey of the available techniques, we perform an in depth analysis of the most promising one, LPU (Learning from Positive and Unlabelled), which turns out to have an F-value and accuracy of about 90%, which means that, given a random text, it has a 90% probability of classifying it correctly. [Copyright &y& Elsevier]

Abstract: Because of the varying and dynamic characteristics of network traffic, such as fast transfer, huge volume, shot-lived, inestimable and infinite, it is a serious challenge for network administrators to monitor network traffic in real time and judge whether the whole network works well. Currently, most of the existing techniques in this area are based on signature training, learning or matching, which may be too complicated to satisfy timely requirements. Other statistical methods including sampling, hashing or counting are all approximate methods and compute an incomplete set of results. Since the main objective of network monitoring is to discover and understand the active events that happen frequently and may influence or even ruin the total network. So in the paper we aim to use the technique of frequent pattern mining to find out these events. We first design a sliding window model to make sure the mining result novel and integrated; then, under the consideration of the distribution and fluidity of network flows, we develop a powerful class of algorithms that contains vertical re-mining algorithm, multi-pattern re-mining algorithm, fast multi-pattern capturing algorithm and fast multi-pattern capturing supplement algorithm to deal with a series of problems when applying frequent pattern mining algorithm in network traffic analysis. Finally, we develop a monitoring system to evaluate our algorithms on real traces collected from the campus network of Peking University. The results show that some given algorithms are effective enough and our system can definitely identify a lot of potentially very valuable information in time which greatly help network administrators to understand regular applications and detect network anomalies. So the research in this paper not only provides a new application area for frequent pattern mining, but also provides a new technique for network monitoring. [ABSTRACT FROM AUTHOR]

In Data Mining, during the preprocessing step, there is a considerable diversity of candidate algorithms to select important features, according to some criteria. This broad availability of algorithms that perform the Feature Selection task gives rise to the difficulty of choosing, a priori , between the algorithms at hand, the most promising one for a particular problem. In this paper, we present the proposal and evaluation of a new architecture for the recommendation of Feature Selection algorithms based on the use of Metalearning. Our framework is very flexible since the user can adapt it to its proper needs. This flexibility is one of the main advantages of our proposal over other approaches in the literature, which involve steps that cannot be adapted to the user’s local requirements. Furthermore, it combines several concepts of intelligent systems, including Machine Learning and Data Mining, with topics derived from expert systems, as user and data-driven knowledge, with meta-knowledge. This set of solutions coupled with leading-edge technologies allows our architecture to be integrated into any information system, which impact on the automation of services and in reducing human effort during the process. Regarding the Metalearning process, our framework considers several types of properties inherent to the data sets, as well as, Feature Selection algorithms based on many information, distance, dependence and consistency measures. The quality of the methods for Feature Selection was estimated according to a multicriteria performance measure, which guided the ranking process of these algorithms for the construction of data metabases. Proposed by the authors of this work, this multicriteria performance measure combines any three measurements on a single one, creating an interesting and powerful tool to evaluate not only FS algorithms but also to assess any context where it is necessary a combination to maximize a measure or minimize it. The recommendation models, represented by decision trees and induced from the training metabases, allowed us to see in what circumstances a Feature Selection algorithm outperforms the other and what aspects of the data present greater influence in determining the performance of these algorithms. Nevertheless, if the user wishes, any other learning algorithm may be used to induce the recommendation model. This versatility is another strong point of this proposal. Results show that with the characterization of data, through statistical, information and complexity measures, it is possible to reach an accuracy higher than 90%. Besides yielding recommendation models that are interpretable and robust to overfitting, the developed architecture is less computationally expensive than approaches recently proposed in the literature. [ABSTRACT FROM AUTHOR]

With the advent of Big Data, data is being collected at an unprecedented fast pace, and it needs to be processed in a short time. To deal with data streams that flow continuously, classical batch learning algorithms cannot be applied and it is necessary to employ online approaches. Online learning consists of continuously revising and refining a model by incorporating new data as they arrive, and it allows important problems such as concept drift or management of extremely high-dimensional datasets to be solved. In this paper, we present a unified pipeline for online learning which covers online discretization, feature selection and classification. Three classical methods—the k -means discretizer, the χ 2 filter and a one-layer artificial neural network—have been reimplemented to be able to tackle online data, showing promising results on both synthetic and real datasets. [ABSTRACT FROM AUTHOR]

In the past two decades, the dimensionality of datasets involved in machine learning and data mining applications has increased explosively. Therefore, feature selection has become a necessary step to make the analysis more manageable and to extract useful knowledge about a given domain. A large variety of feature selection techniques are available in literature, and their comparative analysis is a very difficult task. So far, few studies have investigated, from a theoretical and/or experimental point of view, the degree of similarity/dissimilarity among the available techniques, namely the extent to which they tend to produce similar results within specific application contexts. This kind of similarity analysis is of crucial importance when two or more methods are combined in an ensemble fashion: indeed the ensemble paradigm is beneficial only if the involved methods are capable of giving different and complementary representations of the considered domain. This paper gives a contribution in this direction by proposing an empirical approach to evaluate the degree of consistency among the outputs of different selection algorithms in the context of high dimensional classification tasks. Leveraging on a proper similarity index, we systematically compared the feature subsets selected by eight popular selection methods, representatives of different selection approaches, and derived a similarity trend for feature subsets of increasing size. Through an extensive experimentation involving sixteen datasets from three challenging domains (Internet advertisements, text categorization and micro-array data classification), we obtained useful insight into the pattern of agreement of the considered methods. In particular, our results revealed how multivariate selection approaches systematically produce feature subsets that overlap to a small extent with those selected by the other methods. [ABSTRACT FROM AUTHOR]

Clustering used to discover hidden patterns in unlabeled data sets is an important task in data mining. Therefore, clustering validation applied to evaluating the clustering results has been recognized as one of the vital issues for clustering applications. The existing validation index is often used to evaluate the results rather than guide the clustering process dynamically. However, using the index can automatically adjust the algorithm's operation according to the actual data distribution, thereby improving the algorithm's adaptability. Simulating the division and aggregation process of cells in biology, in this paper, we propose a new two-phase (grouping and merging) cell group clustering algorithm by using a continuous validation and correction mechanism. In the grouping phase, a new clustering internal validation index called Split Index (S I) is utilized to evaluate the cohesion of a cell group continuously, and then a validation and correction mechanism is adopted to validate and split the cell groups so that the S I of each cell group can meet the split threshold ε , finally the cell nucleus of each cell group is determined by finding the sum of the minimum distances from the nucleus to other samples. In the merging phase, the cell group merge method is adopted to merge all the reachable cell groups in a density-reachable manner. Ultimately the clustering problem of arbitrarily distributed samples is completed. Experiments on the synthetic and the UCI Machine Learning Repository 1 1 http://archive.ics.uci.edu/ml/index.php. data set show that the validation index can effectively guide the clustering process, and the algorithm can deal with various data sets, including imbalanced data sets and spherical non-spherical clusters. [Display omitted] • We define a Split Index to evaluate the cohesion of a cell group. • A new method using clustering validation index is applied in clustering process. • Algorithm SICG is proposed based on validation & correction mechanism. • SICG with good result has little dependence on the parameter. [ABSTRACT FROM AUTHOR]

Clustering has been applied in machine learning, data mining and so on, and has received extensive attention. However, since some data has noise or outliers, these noise or outliers easily bring about the objective function with large errors. In this paper, a robust clustering model with ℓ 2 , 1 , ℓ 1 norm and Laplacian regularization (RCLR) is proposed, on which, sparse error matrix is introduced to express sparse noise, and ℓ 1 norm is introduced to alleviate the sparse noise. In addition, the ℓ 2 , 1 norm is also introduced to achieves space robust by virtue of its nice rotation invariance property. Therefore, our RCLR is insensitive to data noise and outliers. More importantly, the Laplacian regularization is introduced into the RCLR to improve the clustering accuracy. In order to solve the optimization objective of clustering problem, we propose an iterative updating algorithm, named alternating direction method of multipliers (ADMM), to update each optimization variable alternatively, and the convergence of the proposed algorithm is also proved in theory. Finally, experimental results on a total of eleven datasets of three types of datasets, elaborate the superiority of this method over six existing classical clustering methods. Three types of datasets include face images dataset, handwritten recognition dataset, and UCI dataset. In particular, our RCLR clustering approach has the best effect on face image dataset. • Propose a joint optimization framework for clustering. • Utilize ℓ 2 , 1 and ℓ 1 to alleviate the influence of outliers and noises. • Prove algorithm convergence from theoretical and practical aspects. • Experimentation on three different types of real datasets. [ABSTRACT FROM AUTHOR]

Feature selection in real-world data mining problems is essential to make the learning task efficient and more accurate. Identifying the best feature selection algorithm, among the many available, is a complex activity that still relies heavily on human experts or some random trial-and-error procedure. Thus, the automated machine learning community has taken some steps towards the automation of this process. In this paper, we address the metalearning challenge of recommending feature selection algorithms by proposing a novel meta-feature engineering model. Our model considers a broad collection of meta-features that enable the study of the relationship between the dataset properties and the feature selection algorithm performance in terms of several criteria. We arrange the input meta-features into eight categories: (i) simple, (ii) statistical, (iii) information-theoretical, (iv) complexity, (v) landmarking, (vi) based on symbolic models, (vii) based on images, and (viii) based on complex networks (graphs). The target meta-features emerge from a multi-criteria performance measure, based on five individual performance indexes, that assesses feature selection methods grounded in information, distance, dependence, consistency, and precision measures. We evaluate our proposal using a recently developed framework that extracts the input meta-features from 213 benchmark datasets, and ranks the assessed feature selection algorithms, to fill in the target meta-features in meta-bases. This evaluation uses five state-of-the-art classification methods to induce recommendation models from meta-bases: C4.5, Random Forest, XGBoost, ANN, and SVM. The results showed that it is possible to reach an average accuracy of up to 90% applying our meta-feature engineering model. This work is the first to use an extensive empirical evaluation to provide a careful discussion of the strengths and limitations of more than 160 meta-features. These meta-features, while designed to aid the task of feature selection algorithm recommendation, can readily be employed in other metalearning scenarios. Therefore, we believe our findings are a valuable contribution to the fields of automated machine learning and data mining, as well as to the feature extraction and pattern recognition communities. • A novel meta-feature engineering model recommends feature selection algorithms. • The proposal obtains promising results from 213 datasets with hit rates of up to 90%. • Some simple, landmarking, image, and graph-based input meta-features highlighted. • A multi-criteria performance measure rigorously assesses candidate algorithms. • Chains of binary or multiclass classifiers can efficiently rank candidate algorithms. [ABSTRACT FROM AUTHOR]

• Discretization is an important process in terms of both machine learning and data mining. • The effect of discretization on ensemble methods was analyzed. • Four Chi-square based discretization algorithms were used. • The Ensemble methods performed better on discrete data sets. Classification is one of the important tasks in data mining and machine learning. Classification performance depends on many factors as well as data characteristics. Some algorithms are known to work better with discrete data. In contrast, most real-world data contain continuous variables. For algorithms working with discrete data, these continuous variables must be converted to discrete ones. In this process called discretization, continuous variables are converted to their corresponding discrete variables. In this paper, four Chi-square based supervised discretization algorithms ChiMerge(ChiM), Chi2, Extended Chi2(ExtChi2) and Modified Chi2(ModChi2) were used. In the literature, the performance of these algorithms is often tested with decision trees and Naïve Bayes classifiers. In this study, differently, four sets of data discretized by these algorithms were classified with ensemble methods. Classification accuracies for these data sets were obtained through using a stratified 10-fold cross-validation method. The classification performance of the original and discrete data sets of the methods is presented comparatively. According to the results, the performance of the discrete data is more successful than the original data. [ABSTRACT FROM AUTHOR]

Abstract: Weighted frequent pattern (WFP) mining is more practical than frequent pattern mining because it can consider different semantic significance (weight) of the items. For this reason, WFP mining becomes an important research issue in data mining and knowledge discovery. However, existing algorithms cannot be applied for incremental and interactive WFP mining and also for stream data mining because they are based on a static database and require multiple database scans. In this paper, we present two novel tree structures IWFPTWA (Incremental WFP tree based on weight ascending order) and IWFPTFD (Incremental WFP tree based on frequency descending order), and two new algorithms IWFPWA and IWFPFD for incremental and interactive WFP mining using a single database scan. They are effective for incremental and interactive mining to utilize the current tree structure and to use the previous mining results when a database is updated or a minimum support threshold is changed. IWFPWA gets advantage in candidate pattern generation by obtaining the highest weighted item in the bottom of IWFPTWA. IWFPFD ensures that any non-candidate item cannot appear before candidate items in any branch of IWFPTFD and thus speeds up the prefix tree and conditional tree creation time during mining operation. IWFPTFD also achieves the highly compact incremental tree to save memory space. To our knowledge, this is the first research work to perform single-pass incremental and interactive mining for weighted frequent patterns. Extensive performance analyses show that our tree structures and algorithms are very efficient and scalable for single-pass incremental and interactive WFP mining. [Copyright &y& Elsevier]

Abstract: In this paper, we propose an algorithm called PITP-Miner that utilizes a projection based approach to mine frequent inter-transaction patterns efficiently. The algorithm only searches for local frequent items in a projected database that stores potential local inter-transaction items and partitions the database into a set of smaller databases recursively. In addition, two pruning strategies are designed to further condense the partitioned databases and thus accelerate the algorithm. Our experiment results demonstrate that the proposed PITP-Miner algorithm outperforms the ITP-Miner and FITI algorithms in most cases. [Copyright &y& Elsevier]

Abstract: One of the most challenging problems in data mining is to develop scalable algorithms capable of mining massive data sets. A novel decision forest learning algorithm named FDF is proposed in this paper to represent multi-level semantic knowledge of the relationship between the data and information implicated. FDF provides their users with just a single set of rules by redefining information gain of information theory, then each tree in the decision forest is constructed in the down-top learning framework, and the number of trees and stopping criteria can be set automatically. When no existing tree match test samples, FDF will build new logical rules for this and thus realize scalable construction process. Empirical studies on a set of natural domains show that decision forest has clear advantages with respect to probabilistic performance. [ABSTRACT FROM AUTHOR]

Abstract: The inclusion of irrelevant, redundant, and inconsistent features in the data-mining model results in poor predictions and high computational overhead. This paper proposes a novel information theoretic-based interact (IT-IN) algorithm, which concerns the relevance, redundancy, and consistency of the features. The proposed IT-IN algorithm is compared with existing Interact, FCBF, Relief and CFS feature selection algorithms. To evaluate the classification accuracy of IT-IN and remaining four feature selection algorithms, Naïve Bayes, SVM, and ELM classifier are used for ten UCI repository datasets. The proposed IT-IN performs better than existing above algorithms in terms of number of features. The specially designed hash function is used to speed up the IT-IN algorithms and provides minimum computation time than the Interact algorithms. The result clearly reveals that the proposed feature selection algorithm improves the classification accuracy for ELM, Naïve Bayes, and SVM classifiers. The performance of proposed IT-IN with ELM classifier is superior to other classifiers. [ABSTRACT FROM AUTHOR]

Abstract: Support Vector Regression (SVR) solves regression problems based on the concept of Support Vector Machine (SVM) introduced by . The main drawback of these newer techniques is their lack of interpretability. In other words, it is difficult for the human analyst to understand the knowledge learnt by these models during training. The most popular way to overcome this difficulty is to extract if–then rules from SVM and SVR. Rules provide explanation capability to these models and improve the comprehensibility of the system. Over the last decade, different algorithms for extracting rules from SVM have been developed. However rule extraction from SVR is not widely available yet. In this paper a novel hybrid approach for extracting rules from SVR is presented. The proposed hybrid rule extraction procedure has two phases: (1) Obtain the reduced training set in the form of support vectors using SVR (2) Train the machine leaning techniques (with explanation capability) using the reduced training set. Machine learning techniques viz., Classification And Regression Tree (CART), Adaptive Network based Fuzzy Inference System (ANFIS) and Dynamic Evolving Fuzzy Inference System (DENFIS) are used in the phase 2. The proposed hybrid rule extraction procedure is compared to stand-alone CART, ANFIS and DENFIS. Extensive experiments are conducted on five benchmark data sets viz. Auto MPG, Body Fat, Boston Housing, Forest Fires and Pollution, to demonstrate the effectiveness of the proposed approach in generating accurate regression rules. The efficiency of these techniques is measured using Root Mean Squared Error (RMSE). From the results obtained, it is concluded that when the support vectors with the corresponding predicted target values are used, the SVR based hybrids outperform the stand-alone intelligent techniques and also the case when the support vectors with the corresponding actual target values are used. [Copyright &y& Elsevier]

Abstract: Algorithms for feature selection in predictive data mining for classification problems attempt to select those features that are relevant, and are not redundant for the classification task. A relevant feature is defined as one which is highly correlated with the target function. One problem with the definition of feature relevance is that there is no universally accepted definition of what it means for a feature to be ‘highly correlated with the target function or highly correlated with the other features’. A new feature selection algorithm which incorporates domain specific definitions of high, medium and low correlations is proposed in this paper. The proposed algorithm conducts a heuristic search for the most relevant features for the prediction task. [Copyright &y& Elsevier]

Data-mining and machine learning must confront the problem of pattern maintenance because data update is a fundamental operation in data management. Most existing data-mining algorithms assume that the database is static, and a database update requires rediscovering all the patterns by scanning the entire old and new data. While there are many efficient mining techniques for data additions to databases, in this paper, we propose a decremental algorithm for pattern discovery when data is deleted from databases. We conduct extensive experiments for evaluating this approach, and illustrate that the proposed algorithm can well model and capture useful interactions within data when the data is decreasing. [Copyright &y& Elsevier]

Abstract: The problem of protein secondary structure prediction is one of the most important problems in Bioinformatics. After the study of this problem for 30 years and more, there have been some breakthroughs. Especially, the introduction of ensemble prediction model and hybrid prediction model makes the accuracy of prediction better, but there is a long distance to induce the tertiary structures from the secondary ones. As one of the extension researches of KDTICM [Bingru, Yang (2004). Knowledge discovery based on theory of inner cognition mechanism and application. Beijing: Electronic Industry Press] theory, this paper proposed a method KAAPRO, which is based on Maradbcm algorithm which is induced by KDD∗ model and combined with CBA, for protein secondary structure prediction. And a gradually enhanced, multi-layer systematic prediction model, compound pyramid model, is proposed. The kernel of this model is KAAPRO. Domain knowledge is used through the whole model, and the physical–chemical attributes are chosen by causal cellular automata. In the experiment, the test proteins used in reference Muggleton et al. (Muggleton, S. H., King, R., Sternberg, M. (1992). Protein secondary structure prediction using logic-based machine learning. Protein Engineering, 5(7), 647–657) are predicted. The structures of amino acids, whose structural traits are obscure, are predicted well by KAAPRO. Hence, the result of this model is satisfying too. [Copyright &y& Elsevier]

Ordinal (i.e., ordered) classifiers are used to make judgments that we make on a regular basis, both at work and at home. Perhaps surprisingly, there have been no comprehensive studies in the scientific literature comparing the various ordinal classifiers. This paper compares the accuracy of five ordinal and three non-ordinal classifiers on a benchmark of fifteen real-world datasets. The results show that the ordinal classifiers that were tested had no meaningful statistical advantage over the corresponding non-ordinal classifiers. Furthermore, the ordinal classifiers that guaranteed monotonic classifications showed no meaningful statistical advantage over a majority-based classifier. We suggest that the tested ordinal classifiers did not properly utilize the order information in the presence of non-monotonic noise. [Copyright &y& Elsevier]