Search

Multi-view clustering has gained soar attention in the field of data mining in recent years. Most multi-view clustering methods are considered to be complete for each view. However, in many applications, some views may contain missing instances, resulting in the incomplete multi-view learning problem. Incomplete views cannot be directly dealt with traditional multi-view clustering methods. In this paper, we propose a One-step multi-view subspace clustering with incomplete views (OMVSC-IV) method based on low-rank matrix factorization to overcome this problem. Specifically, the proposed algorithm uses low-rank matrix factorization to learn a consensus representation matrix, and then combined with the objective function of non-negative embedding and spectral embedding subspace clustering proposed in this paper. The whole process is established into the same objective function for joint optimization and requires no post-processing (e.g., K-means), thus avoiding the defect of being sensitive to initial values. Furthermore, the accuracy, parameter, sensitivity and convergence of the algorithm are studied by experiments. Experimental results on nine data sets show that the proposed algorithm performs superior to some of the latest algorithms.

Recently, a common starting point for solving complex unsupervised image classification tasks is to use generic features, extracted with deep Convolutional Neural Networks (CNN) pretrained on a large and versatile dataset (ImageNet). However, in most research, the CNN architecture for feature extraction is chosen arbitrarily, without justification. This paper aims at providing insight on the use of pretrained CNN features for image clustering (IC). First, extensive experiments are conducted and show that, for a given dataset, the choice of the CNN architecture for feature extraction has a huge impact on the final clustering. These experiments also demonstrate that proper extractor selection for a given IC task is difficult. To solve this issue, we propose to rephrase the IC problem as a multi-view clustering (MVC) problem that considers features extracted from different architectures as different "views" of the same data. This approach is based on the assumption that information contained in the different CNN may be complementary, even when pretrained on the same data. We then propose a multi-input neural network architecture that is trained end-to-end to solve the MVC problem effectively. This approach is tested on nine natural image datasets, and produces state-of-the-art results for IC., Comment: 21 pages, 16 figures, 10 tables, preprint of our paper published in Neurocomputing

Although supervised deep representation learning has attracted enormous attentions across areas of pattern recognition and computer vision, little progress has been made towards unsupervised deep representation learning for image clustering. In this paper, we propose a deep spectral analysis network for unsupervised representation learning and image clustering. While spectral analysis is established with solid theoretical foundations and has been widely applied to unsupervised data mining, its essential weakness lies in the fact that it is difficult to construct a proper affinity matrix and determine the involving Laplacian matrix for a given dataset. In this paper, we propose a SA-Net to overcome these weaknesses and achieve improved image clustering by extending the spectral analysis procedure into a deep learning framework with multiple layers. The SA-Net has the capability to learn deep representations and reveal deep correlations among data samples. Compared with the existing spectral analysis, the SA-Net achieves two advantages: (i) Given the fact that one spectral analysis procedure can only deal with one subset of the given dataset, our proposed SA-Net elegantly integrates multiple parallel and consecutive spectral analysis procedures together to enable interactive learning across different units towards a coordinated clustering model; (ii) Our SA-Net can identify the local similarities among different images at patch level and hence achieves a higher level of robustness against occlusions. Extensive experiments on a number of popular datasets support that our proposed SA-Net outperforms 11 benchmarks across a number of image clustering applications., Comment: arXiv admin note: text overlap with arXiv:2009.05235

Large amounts of training data in machine learning can keep the accuracy high to a certain extent, but the time costs are high because of the exorbitant amount of data and their dimensionality. Therefore, how to simultaneously select the most useful training data set and extract the main features of the samples, especially for image data, are essential problems that urgently need to be solved in the field of large-scale machine learning. Herein, a training sample selection method that is based on the fuzzy c-means clustering algorithm (FCM) is proposed for the problems. It first utilises condensed nearest neighbour (CNN) to make a preliminary selection of training samples. Then, it utilises the FCM to get the centres of the selected data, and, finally, it effectively condenses the sample using a compression parameter. Meanwhile, considering the critical influence of the sample features on the classification model, this paper selects the hierarchical extreme learning machine (H-ELM) model to better solve the classification task. Based on this, the paper presents the FCM-CNN-H-ELM framework for data classification, which combines FCM-Based CNN and H-ELM. The results of the experiments show that the proposed training sample selection method and classification framework can guarantee consistent, even higher, prediction results with a small number of training samples, and significantly reduce the training time.

This paper proposes a method to detect the high-frequency oscillations (HFOs) in epileptic seizure onset zones (SOZs) localization using semi-supervised k-means and mean shift algorithm. Wavelet entropy (WE) and teager energy operator (TEO) are adopted to distinguish HFOs from normal electroencephalogram (EEG). Labeled data are used to initialize the clustering center of semi-supervised k-means algorithm, and unlabeled data are employed to obtain physiological and suspected pathological HFOs. For the suspected pathological HFOs, the mean shift algorithm is used for clustering, and the results are analyzed by the spectral center algorithm to locate SOZs. By comparing the EEG data of five patients with the results of the other three methods, it can be seen that the method proposed in this paper has good sensitivity and specificity, which is helpful for accurate localization before clinical epilepsy surgery.

Probabilistic models have proven their strength to model many natural phenomena as close as possible to reality, in particular, the probabilistic self organizing map (PRSOM) that belongs to the unsupervised learning models. It allows to provide an estimation of the probability density function through the likelihood maximization. This latter function depends on several parameters given by the model architecture. In this context, the aim of this paper is to deal with the architecture choice problem that consists in determining the optimal number of components needed for a better performance. In this paper, we propose a new optimization model that describes the problem above. We present the architecture of PRSOM in a mathematical system of a non linear objective function with mixed variables under linear and quadratic constraints. Due to the complexity of the resolution, we suggest the continuous Hopfield neural network (CHN) that we support by a deep stability analysis. Performance of the proposed model is demonstrated through the dataset clustering.

In the era of big data, all types of data with increasing samples and high-dimensional attributes are demonstrating their important roles in various fields, such as data mining, pattern recognition and machine learning, etc. Meanwhile, machine learning algorithms are being effectively applied in large-scale data processing. This paper mainly reviews the classical dimensionality reduction and sample selection methods based on machine learning algorithms for large-scale data processing. Firstly, the paper provides a brief overview to the classical sample selection and dimensionality reduction methods. Then, it pays attention to the applications of those methods and their combinations with the classical machine learning methods, such as clustering, random forest, fuzzy set, and heuristic algorithms, particularly deep leaning methods. Furthermore, the paper primarily introduces the application frameworks that combine sample selection and dimensionality reduction in the context of two aspects: sequential and simultaneous, which almost all get the ideal results in the processing of the large-scale training data contrasting to the original models. Lastly, we further conclude that sample selection and dimensionality reduction methods are essential and effective for the modern large-scale data processing. In the future work, the machine learning algorithms, especially the deep learning methods, will play a more important role in the processing of large-scale data.

Clustering analysis is to classify elements into categories based on their similarity. Clustering by fast search and find of density peaks (CFSFDP) has been proven to be an effective and novel algorithm, which identifies the centers of clusters with density maxima. However, the performance of CFSFDP is quite sensitive to the estimation of densities, that is exactly the selection of the cutoff distance (dc). In a conventional way, the selection of dc is based on subjective experience. It meets difficulties in finding an appropriate dc, especially for detecting nonspherical clusters, because CFSFDP cannot perform well when there are more than one density peak for one cluster. Besides, another barrier of applying CFSFDP is that manual interaction is always required for making an effective selection of cluster centers. In this paper, a new density-based clustering algorithm, clustering by defining and merging candidates of cluster centers via independence and affinity (CDMC-IA), is proposed. With its strategy, an appropriate value of cutoff distance dc can be well suggested and the robustness of the method itself is enhanced. Moreover, CDMC-IA introduces a new quantity independence to sort and select cluster centers, instead of human based selection from decision graph. Another quantity affinity is also introduced, which well handles multiple density peaks existing in one cluster and is able to assign each data point to the its targeted cluster. The performance of applying conventional clustering methods to benchmark datasets will be compared with the proposed method in this paper.

Neural Networks and Support Vector Machines (SVMs) are two of the most popular and efficient supervised classification models. However, in the context of large datasets many complexity issues arise due to high memory requirements and high computational cost. In the context of the application of Data Mining algorithms, data reduction techniques attempt to reduce the size of training datasets in terms of the number of instances by selecting some of the existing instances or by generating new training instances. The idea is to speed up the application of the data mining algorithm with minimum or no sacrifice in performance. Data reduction techniques have been extensively used in the context of k-Nearest Neighbor classification, a lazy classifier that works by directly using a training dataset rather than building a model. This paper explores the application of data reduction techniques as a preprocessing step before the training step of Neural Networks and SVMs. Furthermore, the paper proposes a new data reduction technique that is based on k-median clustering algorithm. Our experimental results illustrate that, in the case of SVMs, data reduction techniques can effectively reduce the dataset size incurring small performance degradation. In the case of Neural Networks, the performance loss is somewhat greater, for the same data reduction rate, but both SVM and Neural Network models outperform the k-NN approach that is typically used in Data Mining applications.

An important feature characteristic of the data streams in many of today's big data applications is the intrinsic uncertainty, which could happen for both item occurrence and attribute value. While this has already posed great challenges for fundamental data mining tasks such as classification, things are made even more complicated by the fact that completely-labeled examples are usually unavailable in such settings, leaving researchers the only option to learn classifiers on partially-labeled examples on uncertain data streams. Furthermore, there will be concept drift on evolving data streams. To address these challenges, this paper therefore focuses on the study of learning from positive and unlabeled examples (PU) on uncertain data streams. To the best of our knowledge, this paper is the first work to address the uncertainty issue in both item occurrence (occurrence level) and attribute value (attribute level) for the problem of PU learning over streaming data. Firstly, we propose an algorithm to classify positive and unlabeled examples with both uncertainties (PUU). The algorithm extracts reliable positive and negative examples by clustering-based method, and then trains the classifier with Weighted Extreme Learning Machine (Weighted ELM). Secondly, we propose an algorithm of PU learning over uncertain data streams (PUUS). It adopts ensemble model and trains base classification by PUU. In order to detect concept drift, PUUS uses cluster set similarities between the current data block and history data block. We propose different update strategies for different concept drift to adapt to evolving uncertain data streams. Experimental results show that PUUS can effectively classify uncertain data streams with just positive and unlabeled data, while achieving in the meantime good performance in detecting and handling concept drifts.

Being motivated by combining the advantages of hyperplane-based pattern analysis and fuzzy clustering techniques, we present in this paper a fuzzy mix-prototype (FMP) clustering for microarray data analysis. By integrating spherical and hyper-planar cluster prototypes, the FMP is capable of capturing latent data models with both spherical and non-spherical geometric structures. Our contributions of the paper can be summarized into three folds: first, the objective function of the FMP is formulated. Second, an iterative solution which minimizes the objective function under given constraints is derived. Third, the effectiveness of the proposed FMP is demonstrated through experiments on yeast and leukemia data sets.

The problem of subspace clustering which refers to segmenting a collection of data samples approximately drawn from a union of linear subspaces is considered in this paper. Among existing subspace clustering algorithms, low rank representation (LRR) based subspace clustering is a very powerful method and has demonstrated that its performance is good. Latent low rank representation (LLRR) subspace clustering algorithm is an improvement of the original LRR algorithm when the observed data samples are insufficient. The clustering accuracy of LLRR is higher than that of LRR. Recently, Frobenius norm minimization based LRR algorithm has been proposed and its clustering accuracy is higher than that of LRR demonstrating the effectiveness of Frobenius norm as another convex surrogate of the rank function. Combining LLRR and Frobenius norm, a new low rank representation subspace clustering algorithm is proposed in this paper. The nuclear norm in the LLRR algorithm is replaced by the Frobenius norm. The resulting optimization problem is solved via alternating direction method of multipliers (ADMM). Experimental results show that compared with LRR, LLRR and several other state-of-the-art subspace clustering algorithms, the proposed algorithm can get higher clustering accuracy. Compared with LLRR, the running time of the proposed algorithm is reduced significantly.

Interval-valued data arises in situations where it is needed to manage either the uncertainty related to measurements, or the variability inherent to a group rather than an individual. This paper proposes a fuzzy c-means clustering algorithm based on suitable adaptive City-Block and Hausdorff distances with the purpose to cluster interval-valued data. This fuzzy c-means clustering algorithm optimizes explicitly an objective function by alternating three steps aiming to compute the fuzzy cluster representatives, the fuzzy partition, as well as relevance weights for the interval-valued variables. Indeed, most often conventional fuzzy c-means clustering algorithms consider that all variables are equally important for the clustering task. However, in real situations, some variables may be more or less important or even irrelevant for clustering. Due to the use of adaptive City-Block and Hausdorff distances, the paper proposes a fuzzy c-means clustering algorithm that tackles this problem with a step where a relevance weight is automatically learned for each interval-valued variable. Additionally, various tools for the fuzzy partition and fuzzy cluster interpretation of interval-valued data provided by this algorithm is also presented. Experiments with suitable synthetic and real interval-valued datasets demonstrate the robustness and the usefulness of this fuzzy c-means clustering algorithm and the merit of the fuzzy partition and fuzzy cluster interpretation tools.

In this paper, a self-organizing map (SOM) neural network is used to visualize corrective actions of failure modes and effects analysis (FMEA). SOM is a popular unsupervised neural network model that aims to produce a low-dimensional map (typically a two-dimensional map) for visualizing high-dimensional data. With regards to FMEA, it is a popular methodology to identify potential failure modes for a product or a process, to assess the risk associated with those failure modes, also, to identify and carry out corrective actions to address the most serious concerns. Despite the popularity of FMEA in a wide range of industries, two well-known shortcomings are the complexity of the FMEA worksheet and its intricacy of use. To the best of our knowledge, the use of computation techniques for solving the aforementioned shortcomings is limited. The use of SOM in FMEA is new. In this paper, corrective actions in FMEA are described in their severity, occurrence and detect scores. SOM is then used as a visualization aid for FMEA users to see the relationship among corrective actions via a map. Color information from the SOM map is then included to the FMEA worksheet for better visualization. In addition, a Risk Priority Number Interval is used to allow corrective actions to be evaluated and ordered in groups. Such approach provides a quick and easily understandable framework to elucidate important information from a complex FMEA worksheet; therefore facilitating the decision-making tasks by FMEA users. The significance of this study is two-fold, viz., the use of SOM as an effective neural network learning paradigm to facilitate FMEA implementations, and the use of a computational visualization approach to tackle the two well-known shortcomings of FMEA.

The paper investigates application of several methods of feature selection to identification of the most important genes in autism disorder. The study is based on the expression microarray of genes. The applied methods analyze the importance of genes on the basis of different principles of selection. The most important step is to fuse the results of these selections into common set of genes, which are the best associated with autism. These genes may be treated as the biomarkers of this disorder and used in early prediction of autism. The paper proposes and compares three different methods of such fusion: purity of the clusterization space, application of genetic algorithm and random forest in the role of integrator. The numerical experiments are concerned with the identification of the most important biomarkers and their application in autism recognition. They show the applied fusion strategy of many independent selection methods leads to the significant improvement of the autism recognition rate.

The new dynamic crossover and entropy-based two-combination mutation operations are constructed to prevent the convergence of the algorithms to a local optimum by adaptively modifying the probabilities of crossover and mutation as well as mutation step size according to dynamic adjusting strategies and judging criterions.An improved self-adaptive cellular genetic algorithm (IDCGA) is presented for a more efficient search by combining the Arnold cat map with modified evolution rule, as well as the constructed dynamic crossover and the entropy based two-combination mutation operators.Two novel adaptive fuzzy clustering algorithms based on IDCGA, referred to as IDCGA-FCM and IDCGA2-FCM, are proposed in this paper. The first one is a standalone form for fuzzy clustering on the basis of IDCGA. The second one is a hybrid method based on FCM and IDCGA which takes advantage of the merits of both algorithms.The experimental results showed that the presented algorithms have high efficiency and accuracy. With an aim to overcome low efficiency and improve the performance of fuzzy clustering, two novel fuzzy clustering algorithms based on improved self-adaptive cellular genetic algorithm (IDCGA) are proposed in this paper. The new dynamic crossover and entropy-based two-combination mutation operations are constructed to prevent the convergence of the algorithms to a local optimum by adaptively modifying the probabilities of crossover and mutation as well as mutation step size according to dynamic adjusting strategies and judging criterions. Arnold cat map is employed to initialize population for the purpose of overcoming the sensitivity of the algorithms to initial cluster centers. A modified evolution rule is introduced to build a dynamic environment so as to explore the search space more effectively. Then a new IDCGA that combined these three processes is used to optimize fuzzy c-means (FCM) clustering (IDCGA-FCM). Furthermore, an optimal-selection-based strategy is presented by the golden section method and then a hybrid fuzzy clustering method (IDCGA2-FCM) is developed by automatically integrating IDCGA with optimal-selection-based FCM according to the variation of population entropy. Experiments were performed with six synthetic datasets and seven real-world datasets to compare the performance of our IDCGA-based clustering algorithms with FCM, other GA-based and PSO-based clustering methods. The results showed that the presented algorithms have high efficiency and accuracy.

High-dimensional data in the real world often resides in low-dimensional subspaces. The state-of-the-art methods for subspace segmentation include Low Rank Representation (LRR) and Sparse Representation (SR). The former seeks the global lowest rank representation but restrictively assumes the independence among subspaces, whereas the latter seeks the clustering of disjoint or overlapped subspaces through locality measure, which may cause failure in the case of large noises. To this end, a Low Rank subspace Sparse Representation framework, hereafter referred to as LRSR, is proposed in this paper to recover and segment embedding subspaces simultaneously. Three major contributions can be claimed in this paper: First, a clean dictionary is constructed by optimizing its nuclear norm, low-rank-sparse coefficient matrix obtained using linearized alternating direction method (LADM). Second, both the convergence proof and the complexity analysis are given to prove the effectiveness and efficiency of our proposed LRSR algorithm. Third, the experiments on synthetic data and two benchmark datasets further verify that the LRSR enjoys the capability of clustering disjoint subspaces as well as the robustness against large noises, thanks to its considerations of both global and local subspace information. Therefore, it has been demonstrated in this research that our proposed LRSR algorithm outperforms the state-of-the-art subspace clustering methods, verified by both theoretical analysis and the empirical studies.

In the traditional Support Vector Machine Plus (SVM+), the grouping method has great randomness and only takes into account part of the structural information of the dataset. In order to overcome these shortcomings, in this paper, we propose a novel framework termed as FCSVM+ to improve the performance of SVM+ by combining clustering technique and feature selection. The new framework strategy is expected to not only take fully into account the structural information of the training data, but also partition the training data into more meaningful groups. To prove the advantage of the framework, in particular, we adopt two simplest feature selection methods, i.e. F-score and Laplacian score methods, to select the features, then apply a recently proposed clustering technique to get a better partition of training data by the selected features, in which the number of clusters could be found automatically. Three major contributions of this paper can be concluded as: (1) improving the performance of the existing SVM+ classifier; (2) extracting the potential structural information of the training data by using more feature attributes instead of one; (3) replacing the truncation method in SVM+ with a clustering technique. The comprehensive experimental results on the UCI benchmark datasets illustrate the validity and advantage of our approach.

We propose a robust object tracking algorithm based on local region sparse appearance model in this paper. In this algorithm, the object is divided into several sub-regions, and the sparse dictionaries are obtained by clustering in each sub-region. Therefore spatial structure information of the object can be captured well, and the change of object appearance can be also resisted effectively. First, the object is divided into many small patches. Then the object is divided into several sub-regions according to patch distribution again. The establishment of object dictionary base is based on combination of the dictionaries from all the sub-regions, and then space alignment between different parts of the object can be achieved. Meanwhile, noise removal and other operations in the existing sparse reconstruction error maps are performed to retain valuable information. In the updating framework, a novel flexible template set update mechanism is introduced in this paper. In this update mechanism, valuable object samples will be put into the template set. If samples are not valuable, they should not be put into the template set, even when the template set is not full. Then we use patch sparse coefficient histogram of updated templates to extract time domain information of the object in the form of weighted sum. Therefore, it can provide a reliable template basis for obtaining good candidate object. In addition, when tracking result deviates from the actual position of the object, we use a dynamic sub-region resampling method based on cosine angle to correct the position deviation timely. Therefore this method can effectively prevent the object from being completely lost. Both qualitative and quantitative evaluations on challenging video sequences demonstrate that the proposed tracking algorithm performs favorably against several state-of-the-art methods.

With the rapid accumulation of high dimensional data, dimensionality reduction plays a more and more important role in practical data processing and learning tasks. This paper studies semi-supervised dimensionality reduction using pairwise constraints. In this setting, domain knowledge is given in the form of pairwise constraints, which specifies whether a pair of instances belong to the same class (must-link constraint) or different classes (cannot-link constraint). In this paper, a novel semi-supervised dimensionality reduction method called Adaptive Semi-Supervised Dimensionality Reduction with Sparse Representation (ASSDR-SR) is proposed, which can get the optimized low dimensional representation of the original data by adaptively adjusting the weights of the pairwise constraints and simultaneously optimizing the graph construction using the ?1 graph of sparse representation. Experiments on clustering and classification tasks show that ASSDR-SR is superior to some existing dimensionality reduction methods.

Clustering ensemble is an important part of ensemble learning. It aims to study and integrate multiple clustering results from different clustering algorithms or same algorithm with different initial parameters for the same dataset. CHAMELEON is a hierarchical clustering algorithm which can discover natural clusters of different shapes and sizes as the result of its merging decision dynamically adapts to the different clustering model characterized. Inspired by the idea of CHAMELEON, the paper proposes a novel clustering ensemble models including semi-supervised method and discusses its application in fault diagnosis of high speed train (HST) running gear. The contributions of this paper include: constructing a sparse graph via the similarity matrix which aggregates multiple clustering results; partitioning the sparse graph (vertex=object, edge weight=similarity) into a large number of relatively small sub-clusters; obtaining the final clustering partition by merging these sub-clusters repeatedly. The experimental results demonstrate that our method outperforms some of state-of-the-art ensemble algorithms regarding the accuracy and stability and recognizes fault patterns of HST running gear effectively.

As a classical partitional clustering algorithm, k-means algorithm is sensitive to initial centroids and may malfunction when dealing with datasets which contain clusters with different scales and densities. To improve the effectiveness of k-means algorithm, an outlier factor based partitional clustering analysis method is presented in this paper. Outlier factor is usually used to indicate the degree of an object to be abnormal in the dataset. For the proposed method, it is used to find the core objects. And then the Must-link constraints is generated to put the neighboring core objects into the same cluster. First, a similar-density-array-based outlier factor is proposed to find the core objects in the dataset. Then the neighboring core objects are distributed into the same sub-cluster. The sub-clusters are treated as the representative objects and these representative objects are then clustered following the process of the traditional k-means algorithm. Finally, the non-core objects are assigned to their nearest clusters, respectively. The experiments are performed on four datasets from UCI Machine Learning Repository and a field dataset from a ball mill pulverizing system. The experimental results verify that the effectiveness of our algorithm is high. HighlightsNew method to combine outlier factor with k-means algorithm, and local information is effectively used.The proposed method is compared with two classical constrained k-means algorithms.Besides UCI datasets, a field dataset from a ball mill pulverizing system is used in the experiment.An outlier factor is proposed in the paper and is compared with LOF and COF in the experiment.The impact of the parameter of the outlier factors are tested and discussed in detail in the experiment.

During the last decade, multi-label learning has attracted the attention of more and more researchers in machine learning field due to wide real-world applications. Existing approaches often predict an unseen example for all labels based on the same feature vector. However, this strategy might be suboptimal since different labels usually depend on different aspects of the feature vector. Furthermore, for each label there is close relationship between positive and negative instances, which is quite informative for classification. In this paper, we propose a new algorithm called ML-DFL, which trains a model for each label with newly constructed discriminative features. In order to form these features, we also propose a spectral clustering algorithm SIA to find the closely located local structures between positive and negative instances, which are assumed to be of more discriminative information, and then transform the original data set by consulting the clustering results in a simple but effective way. Comprehensive experiments are conducted on a collection of benchmark data sets. The results clearly validate the superiority of ML-DFL to various competitors. This paper proposes a novel multi-label learning algorithm, Multi-label Learning with Discriminative Features for each Label (ML-DFL), based on binary relevance by exploiting the correlations between positive and negative instances for each label.To investigate the correlations as stated in Point 1, we propose a new spectral clustering algorithm Spectral Instance Alignment (SIA).Substantial experiments across six data sets from three different application domains validate the superiority of our proposed ML-DFL to five competitors.

In the first part of this paper, we present a method to build affinity matrices for spectral clustering from a density estimator relying on K-means with subbagging procedure. The approach is anchored in the theoretical works of Wong (1980, 1982a, b) [13] , [14] , [15] on the asymptotic properties of K-means as a density estimation method. The subbagging procedure is introduced to improve the density estimate accuracy. The behavior of the proposed method is analyzed on diverse data sets and two new mechanisms are suggested to improve clustering results on non-convex data. In the second part of the paper, we establish a link between the presented method and the evidence accumulation clustering (EAC) approach by showing that a normalized version of the density-based similarity matrix is approximately equal to a normalized version of the co-association matrix. The co-association matrix provides the co-occurrence probability of data pairs assigned to a same cluster over multiple K-means clustering partitions. Experimental results on artificial and real data demonstrate the effectiveness of the method and provide empirical support for the established link.

This paper explores the performance of Ensembles of Extreme Learning Machine classifiers for hyperspectral image classification and segmentation in a semisupervised and spatially regularized process. The approach assumes that we have available only a small training set of labeled samples, which we enrich with a set of guessed labelings on selected samples from the vast pool of unlabeled image pixels. Selection and label guessing is conditioned to an unsupervised classification of the image pixel spectra, and to the spatial proximity to the labeled samples in the image domain. Unlabeled pixels falling in the spatial neighborhood of a labeled training sample, and belonging to the same unsupervised class, acquire its label. Unsupervised classification can be performed by any clustering technique, in this paper we have resorted to the classical K-means. The classifier built from the enriched training dataset is applied to the entire hyperspectral image. Finally, we perform a spatial regularization of the classification label image, maximizing a rather general prior smoothness criterion, by the selection of the most frequent class in each pixel neighborhood. This paper reports experiments with homogeneous ensembles of ELM, rELM, and OP-ELM classifiers, including a sensitivity analysis over the ensemble size and the number of hidden nodes. Computational experiments on four well known benchmarking hyperspectral images give state-of-the-art results.

The paper presents a novel resampling method for binary classification problems on imbalanced datasets. Imbalanced datasets are frequently found in many industrial applications: for instance, the occurrence of particular product defects, the diagnosis of severe diseases in a series of patients or machine faults are rare events whose detection is of utmost importance. In this paper a new resampling method is proposed combining an oversampling and an undersampling technique. Several tests have been developed aiming at assessing the efficiency of the proposed method. Four classifiers based, respectively, on Support Vector Machine, Decision Tree, labelled Self-Organizing Map and Bayesian Classifiers have been developed and applied for binary classification on the following four datasets: a synthetic dataset, a widely used public dataset and two datasets coming from industrial applications. The results that have been obtained in the tests are presented and discussed in the paper; in particular, the performances that are achieved by the four classifiers through the proposed novel resampling approach have been compared to the ones that are obtained, without any resampling, through a widely applied and well known resampling technique, i.e. the classical SMOTE approach, and through another approach coupling informed SMOTE-based oversampling and informed clustering-based undersampling.

An unsupervised image segmentation method for natural images is proposed in this paper. We assume that texture features in natural images are distributed as a mixture of Gaussians. In order to cluster the extracted feature vectors, we modify a clustering algorithm based on Bayesian Ying-Yang (BYY) harmony learning theory with Dirichlet-Normal-Wishart prior. This algorithm can determine the number of components automatically during the clustering procedure, as long as we give a large enough initial component number. Our works in this paper have presented a complete pipeline of clustering-based image segmentation including feature extraction, robust feature clustering and methodological effective post processing. The experiments reported in this paper demonstrate that the proposed method is efficient (in terms of visual evaluation and quantitative performance measures) and performs competitively compared to the existing state-of-the-art segmentation methods.

This paper extends the use of recently introduced Self-Organizing Time Map (SOTM) by pairing it with classical cluster analysis. The SOTM is an adaptation of the Self-Organizing Map for visualizing dynamics in cluster structures. While enabling visual dynamic clustering of temporal and cross-sectional patterns, the stand-alone SOTM lacks means for objectively representing temporal changes in cluster structures. This paper combines the SOTM with clustering and illustrates the usefulness of second-level clustering for representing changes in cluster structures in an easily interpretable format. This provides means for identification of changing, emerging and disappearing clusters over time. Experiments are performed on two toy datasets and two real-world datasets. The first real-world application explores evolution dynamics of European banks before and during the global financial crisis. Not surprisingly, the results indicate a build-up of risks and vulnerabilities throughout the European banking sector prior to the start of the crisis. The second application identifies the cyclicality of currency crises through changes in the most vulnerable clusters.

Radial basis function (RBF) neural networks have been broadly used for classification and interpolation regression. So the idea for trying to develop new learning algorithms for getting better performance of RBF neural networks is an interesting subject. This paper presents a new learning method for RBF neural networks. A novel Particle Swarm Optimization (PSO) has been applied in the proposed method to optimize the Optimum Steepest Decent (OSD) algorithm. The OSD algorithm could be used in applications where need real-time capabilities for retraining neural networks. To initialize the RBF units more accurately, the new approach based on PSO has been developed and compared with a Conventional PSO clustering algorithm. The obtained results have shown better and same network response in fewer train iterations which is essential for fast retraining of the network. The PSO-OSD and Three-phased OSD algorithms have been applied on five benchmark problems and the results have been compared. Finally, employing the proposed method in a real-time problem has shown interesting outcomes as have come out in this paper.

Due to the limitation of the local spatial information in an image, fuzzy c-means clustering algorithms with the local spatial information cannot obtain the satisfying segmentation performance on the image heavily contaminated by noise. In order to compensate this drawback of the local spatial information, an effective kind of non-local spatial information is extracted from the image in this paper. In the acquisition of non-local spatial information, the filtering degree parameter h is a very crucial parameter and needs to be set appropriately. Instead of using a single h value for all the pixels, the calculation of the adaptive parameter h for each pixel is done by studying the statistical characteristics in its search window. Therefore, the non-local spatial information obtained by using the adaptive h value for each pixel is called self-tuning non-local spatial information. In this paper, two novel fuzzy clustering algorithms using the self-tuning non-local spatial information are proposed. In the first algorithm framework, a spatial constraint term by utilizing the self-tuning non-local spatial information for each pixel is defined and then introduced into the objective function of FCM. This algorithm is called fuzzy c-means clustering algorithm with self-tuning non-local spatial information (FCM_SNLS). In the second algorithm framework, a novel gray level histogram is constructed by using the self-tuning non-local spatial information for each pixel, and then clustering is performed on this gray level histogram. This algorithm is called fast fuzzy c-means clustering algorithm with self-tuning non-local spatial information (FFCM_SNLS). Experimental results show that these two proposed methods are not only more effective than fuzzy clustering algorithms with the local spatial information in noise suppression and edge preservation, but also more robust than fuzzy clustering algorithms with the non-local spatial information.

Semi-supervised clustering under pairwise constraints (i.e. must-links and cannot-links) has been a hot topic in the data mining community in recent years. Since pairwise constraints provided by distinct domain experts may conflict with each other, a lot of research work has been conducted to evaluate the effects of noise imposing on semi-supervised clustering. In this paper, we introduce elite pairwise constraints, including elite must-link (EML) and elite cannot-link (ECL) constraints. In contrast to traditional constraints, both EML and ECL constraints are required to be satisfied in every optimal partition (i.e. a partition with the minimum criterion function). Therefore, no conflict will be caused by those new constraints. First, we prove that it is NP-hard to obtain EML or ECL constraints. Then, a heuristic method named Limit Crossing is proposed to achieve a fraction of those new constraints. In practice, this new method can always retrieve a lot of EML or ECL constraints. To evaluate the effectiveness of Limit Crossing, multi-partition based and distance based methods are also proposed in this paper to generate faux elite pairwise constraints. Extensive experiments have been conducted on both UCI and synthetic data sets using a semi-supervised clustering algorithm named COP-KMedoids. Experimental results demonstrate that COP-KMedoids under EML and ECL constraints generated by Limit Crossing can outperform those under either faux constraints or no constraints.

In handwritten character recognition, it is a significant step to segment a text line into characters. The unsupervised clustering is a common approach for this task. However, due to the strong overlapping and touch among characters, the separation boundaries between two characters are usually nonlinear, which leads to the failure of the widely used clustering methods such as k-means. To tackle this problem, this paper proposes a new handwritten character segmentation method based on nonlinear clustering methods. In the proposed approach, we first segment the entire text line into strokes, the similarity matrix of which is computed according to stroke gravities. Then, the nonlinear clustering methods are performed on this similarity matrix to obtain cluster labels for these strokes. According to the obtained cluster labels, the strokes are combined to form characters. In this paper, we consider two nonlinear clustering methods, namely, spectral clustering based on Normalized cut (Ncut) and kernel clustering based on Conscience On-Line Learning (COLL). Whereby, two segmentation approaches are proposed with the one using Ncut termed SegNcut, and the one using COLL termed SegCOLL. Experiments on four databases are conducted to demonstrate the effectiveness of our SegNcut and SegCOLL approaches.

This paper addresses the three important issues associated with competitive learning clustering, which are auto-initialization, adaptation to clusters of different size and sparsity, and eliminating the disturbance caused by outliers. Although many competitive learning methods have been developed to deal with some of these problems, few of them can solve all the three problems simultaneously. In this paper, we propose a new competitive learning clustering method termed energy based competitive learning (EBCL) to simultaneously tackle these problems. Auto-initialization is achieved by extracting samples of high energy to form a core point set, whereby connected components are obtained as initial clusters. To adapt to clusters of different size and sparsity, a novel competition mechanism, namely, size-sparsity balance of clusters (SSB), is developed to select a winning prototype. For eliminating the disturbance caused by outliers, another new competition mechanism, namely, adaptive learning rate based on samples' energy (ALR), is proposed to update the winner. Data clustering experiments on 2000 simulated datasets comprising clusters of different size and sparsity, as well as with outliers, have been performed to verify the effectiveness of the proposed method. Then we apply EBCL to automatic color image segmentation. Comparison results show that the proposed EBCL outperforms existing competitive learning algorithms.

This paper develops a hybrid model for single point short term traffic flow forecasting in an urban traffic network. The hybrid model consists of two main modules: a fuzzy input fuzzy output filter (FIFO-filter) and a multi-layer feed-forward artificial neural network architecture optimized using evolution strategies (MLFN-ES). The FIFO-filter performs the data clustering operation and provides a rough forecasted prediction value based on the input data to the MLFN-ES associated with each cluster for modeling the input-output relation to provide accurate short term forecast value. The performance of the proposed model is demonstrated by predicting the traffic flow for an intersection in the central business district (CBD) area of Singapore. The hybrid model proposed in this paper gave a mean absolute percentage error (MAPE) of 8.35% on weekdays and 9.73% on weekends for the test data. A comparison analysis shows improved performance of the proposed hybrid method in short term traffic prediction over popular approaches like ARIMA and artificial neural network based systems.