Search

Automatic clustering and dimension reduction are two of the most intriguing topics in the field of clustering. Affinity propagation (AP) is a representative graph-based clustering algorithm in unsupervised learning. However, extracting features from high-dimensional data and providing satisfactory clustering results is a serious challenge for the AP algorithm. Besides, the clustering performance of the AP algorithm is sensitive to preference. In this paper, an improved affinity propagation based on optimization of preference (APBOP) is proposed for automatic clustering on high-dimensional data. This method is optimized to solve the difficult problem of determining the preference of affinity propagation and the poor clustering effect for non-convex data distribution. First, t-distributed stochastic neighbor embedding is introduced to reduce the dimensionality of the original data to solve the redundancy problem caused by excessively high dimensionality. Second, an improved hybrid equilibrium optimizer based on the crisscross strategy (HEOC) is proposed to optimize preference selection. HEOC introduces the crisscross strategy to enhance local search and convergence efficiency. The benchmark function experiments indicate that the HEOC algorithm has better accuracy and convergence rate than other swarm intelligence algorithms. Simulation experiments on high-dimensional and real-world datasets show that APBOP has better effectiveness. [ABSTRACT FROM AUTHOR]

K-nearest neighbors search (K-NNS) is a fundamental problem in many areas of machine learning and data mining. In an attempt to solve NNS problems by locality-sensitive hashing (LSH)-based algorithms and avoid the F1-trap, in this paper, we theoretically prove the necessity of maintaining close edges of each dimension, the independence of the partition location and the precision of retrieval. Based on the theories, a new method is proposed for constructing the hash function family. Then, a volume-balancing locality-sensitive hashing (VBLSH) algorithm is implemented. In this algorithm, the problem of constructing the hash function family is transformed into an optimization problem under a fixed number of hypercubes. By solving the optimization problem, different hyperplane intervals are set for each dimension to ensure that the hash functions can evenly segment the data space, where each segmented hypercube has a close volume and close side lengths. Finally, the analysis of the algorithm shows that the VBLSH algorithm has a high index coding efficiency and hash code calculation efficiency. The experimental results show that the VBLSH algorithm can simultaneously take into account the efficiency, precision and recall rate, and avoid the F1-trap; it shows excellent effectiveness on 4 datasets. [ABSTRACT FROM AUTHOR]

Rule learning is a special type of machine learning approaches, and its key advantage is the generation of interpretable models, which provides a transparent process of showing how an input is mapped to an output. Traditional rule learning algorithms are typically based on Boolean logic for inducing rule antecedents, which are very effective for training models on data sets that involve discrete attributes only. When continuous attributes are present in a data set, traditional rule learning approaches need to employ crisp intervals. However, in reality, problems usually show shades of grey, which motivated the development of fuzzy rule learning approaches by employing fuzzy intervals for handling continuous attributes. While a data set contains a large portion of discrete attributes or even no continuous attributes, fuzzy approaches cannot be used to learn rules effectively, leading to a drop in the performance. In this paper, a multi-stage approach of mixed rule learning is proposed, which involves strategic combination of both traditional and fuzzy approaches to handle effectively various types of attributes. We compare our proposed approach with existing algorithms of rule learning. Our experimental results show that our proposed approach leads to significant advances in the performance compared with the existing algorithms. [ABSTRACT FROM AUTHOR]

Multiple-Instance Learning (MIL) refers to the problem wherein each object is a bag consisting of multiple instances and only the bags' labels are provided. MIL data can contain irrelevant, redundant, and noisy components, which makes feature-extraction preprocessing essential for performance improvement. In this paper, we propose a Multiple-Instance Feature Extraction (MIFE) framework to design algorithms at both the bag and instance levels based on the Maximum Trace-Difference criterion, which simultaneously maximizes between-class scattering and minimizes within-class scattering. MIFE not only treats the existing Multiple-Instance Discriminant Analysis algorithm as an instance-level realization but also enables us to adopt different bag-level distances to design corresponding bag-level algorithms. In particular, we introduce the Class-to-Bag (C2B) and Bag-to-Bag (B2B) distances into the MIFE framework and obtain the MIFE-C2B and MIFE-B2B algorithms, respectively. The experimental results show that both MIFE-C2B and MIFE-B2B obtain competitive classification performance, and MIFE-B2B obtains the best performance on most tested datasets. The dimensionality reduction results show that both MIFE-C2B and MIFE-B2B obtain their best performance with no more than approximately 30% of the original dimensions on most tested datasets. [ABSTRACT FROM AUTHOR]

In this paper, a dynamic mentoring scheme along with a self-regulation scheme have been incorporated in the standard Particle Swarm Optimization (PSO) algorithm to empower the searching particles with human-like characteristics. The algorithm is referred to as a Dynamic Mentoring and Self-Regulation based Particle Swarm Optimization (DMeSR-PSO) algorithm. Based on their experiences, the particles are divided into three groups, viz., the mentor group, the mentee group and the independent learner group where the number of particles in each group is dynamically changing in every iteration. In human learning psychology, mentoring is regarded as a powerful and effective learning process and independent learners are the ones who do not need mentoring and are capable of performing self-regulation of their own knowledge. Therefore, the particles in each of the above three groups have different learning strategies for their velocity updates where the mentors are equipped with a strong self-belief based search, the mentees are taking guidance from the mentors and the independent learners employ self-perception strategy. The DMeSR-PSO algorithm has been extensively evaluated using the simple unimodal and multimodal benchmark functions from CEC2005, more complex shifted and rotated benchmark functions fromCEC2013 and also based on eight real- world problems from CEC2011. The results have been compared with six state-of-the-art PSO variants and five meta-heuristic algorithms for the CEC2005 problems. Further, a comparative analysis on CEC2013 benchmark functions with different PSO variants has also been presented. Finally, DMeSR-PSO's performance on the real-world problems is compared with the top two algorithms from the CEC2011 competition. The results indicate that the proposed learning strategies help DMeSR-PSO to achieve faster convergence and provide better solutions in most of the problems with a 95% confidence level, yielding an effective optimization algorithm for real-world applications. [ABSTRACT FROM AUTHOR]

Data clustering aims to segment a database into groups of objects based on the similarity among these objects. Due to its unsupervised nature, the search for a good-quality solution can become a complex process. There is currently a wide range of clustering algorithms, and selecting the best one for a given problem can be a slow and costly process. In 1976, Rice formulated the Algorithm Selection Problem (ASP), which postulates that the algorithm performance can be predicted based on the structural characteristics of the problem. Meta-learning brings the concept of learning about learning; that is, the meta-knowledge obtained from the algorithm learning process allows the improvement of the algorithm performance. Meta-learning has a major intersection with data mining in classification problems, in which it is normally used to recommend algorithms. The present paper proposes new ways to obtain meta-knowledge for clustering tasks. Specifically, two contributions are explored here: (1) a new approach to characterize clustering problems based on the similarity among objects; and (2) new methods to combine internal indices for ranking algorithms based on their performance on the problems. Experiments were conducted to evaluate the recommendation quality. The results show that the new meta-knowledge provides high-quality algorithm selection for clustering tasks. [ABSTRACT FROM AUTHOR]