Search

Clustering and classification-based pattern recognition techniques are widely used in various domains. While many approaches exist to perform these tasks, it remains a difficult process to perform clustering and classification simultaneously for particular datasets. In this paper, a method based on k-nearest neighbor (KNN) is presented to classify the dataset with PSO optimized k-medoids clustering. Initial clustering with k-medoids algorithm divides the dataset into smaller and disjoint clusters featuring similarity within clusters and dissimilarity with members of other clusters. Particle swarm optimization (PSO) is an evolutionary algorithm mainly used to optimize the issues in several research areas including data analytics. The fitness function of the PSO approach mentioned in this paper is based on inertia weights that identify the particles with the best positions and velocities for optimization. Additionally, PSO uses a novel elitism concept that allows massive searching capability between the swarm of particles to achieve a better convergence rate. Because of this property, it can be applied throughout different machine learning fields. Moreover, the KNN classifier outperforms the classification task in terms of classifying the optimized particles with high accuracy. The performance of the proposed technique is evaluated by experimenting with datasets taken from open sources. The simulation results revealed that the performance of the proposed method is better than the existing methods in terms of effective clustering as well as accurate classification. [ABSTRACT FROM AUTHOR]

In many machine learning applications and algorithms, the algorithm performance and accuracy are highly dependent on the metric used to measure the distance between different samples. Therefore, learning a distance metric specific to the data can improve these algorithms' performance. This paper proposes an unrestricted deep metric learning framework based on neural networks' interaction for learning metrics in latent space. The proposed method is inspired by generative neural nets (GANs), in which two neural nets are working together to learn true data distribution. In our method, one network plays the role of a supervisor for another network, a feature learning auto-encoder. Its task is to learn transformation to latent space in which data have more meaningful distance and separability. i.e., the supervisor gets the output of the auto-encoder and sends feedback to modify its weights. They interact with each other interleavingly. Several experiments were conducted on four datasets, such as MNIST, GISETTE, Winnipeg Cropland Classification (WCC), and swarm behavior, from different application domains, to evaluate the proposed method's performance. The results show that we can force auto-encoder to learn label information to project data into a latent space with better separability by using our approach. In addition to better class discrimination, the proposed method is far faster than normal auto-encoders during feature learning and has much less training time in the classification phase. [ABSTRACT FROM AUTHOR]

This paper addresses learning in complex scenarios involving imbalance and overlap. We propose a novel measure, the Augmented R-value, for estimating the level of overlap in the data. It improves an existing model-based measure, by including the data imbalance in the estimation process. We provide both a theoretical demonstration and empirical validations of the new metric’s efficacy in estimating the overlap level. Another contribution of the present paper is to propose a collection of meta-features to be used in conjunction with a meta-learning strategy for predicting the most suitable classifier for a given problem. The evaluations performed on a well-known collection of benchmark problems have shown that the meta-learning approach achieves superior results to the manual classifier selection process normally carried out by data scientists. The analysis of the results obtained by the meta-feature selection step has confirmed the power of the Augmented R-value in predicting the expected performance of classifiers in such complex classification scenarios. Also, we found that the overlap is a more serious factor affecting the performance of classifiers than imbalance. [ABSTRACT FROM AUTHOR]

Advantages of machine learning algorithms are compared in terms of "generalization," which means how accurate they perform upon unseen data. There are many applications in which test data are available (obviously without target). A central question with regard to "generalization" is as follows: Would it be possible to deploy test feature vectors in the training phase? For test data, nonetheless, there is no target—this is the paradox. Transductive learning is a form of learning that uses test data in addition to train data in the training phase. Using decision tree as the base model reveals many errors occur on those data, the feature values of which, are very close to splitting values of decision tree's nodes. This observation is the essence of answering the aforementioned paradox in this study. This paper proposes a combination of decision tree and a novel Fisher discriminant analysis, in which the proposed modified FDA has an embedded mechanism using test data as a shortcut to generalization. This combination further arranged in a boosting system with weights on both train and test data. Experiments are evaluated on 21 datasets from various fields. The results decisively demonstrate the promising performance of the proposed method. [ABSTRACT FROM AUTHOR]

Stable augmented reality applications consist of an accurate registration supported by a robust tracking module. In outdoor locations, the changing environmental and light conditions compromise this tracking. Reliable descriptors under unsettled conditions are essential for this process. The most used descriptors have this distinctive capacity, but computers and mobile devices process them in a long time frame. This paper investigates a new lightweight environment dedicated descriptor (EDD) trained with a machine-learning algorithm. The descriptor analyzes the scene characteristics with elements that can be computed fast and that have distinctive information about the selected area. The complete descriptor is used for semantic feature extraction with the aid of a trained random forest classifier. The descriptor is compared with the most popular descriptors—with respect to speed, accuracy, and invariance to illumination changes, scale, affine transformation, and rotation—and the results show that it is faster and in most cases equally reliable. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF) is an unsupervised learning method for decomposing high-dimensional nonnegative data matrices and extracting basic and intrinsic features. Since image data are described and stored as nonnegative matrices, the mining and analysis process usually involves the use of various NMF strategies. NMF methods have well-known applications in face recognition, image reconstruction, handwritten digit recognition, image denoising and feature extraction. Recently, several projective NMF (P-NMF) methods based on positively constrained projections have been proposed and were found to perform better than the standard NMF approach in some aspects. However, some drawbacks still affect the existing NMF and P-NMF algorithms; these include dense factors, slow convergence, learning poor local features, and low reconstruction accuracy. The aim of this paper is to design algorithms that address the aforementioned issues. In particular, we propose two embedded P-NMF algorithms: the first method combines the alternating least squares (ALS) algorithm with the P-NMF update rules of the Frobenius norm and the second one embeds ALS with the P-NMF update rule of the Kullback-Leibler divergence. To assess the performances of the proposed methods, we conducted various experiments on four well-known data sets of faces. The experimental results reveal that the proposed algorithms outperform other related methods by providing very sparse factors and extracting better localized features. In addition, the empirical studies show that the new methods provide highly orthogonal factors that possess small entropy values. [ABSTRACT FROM AUTHOR]

Aiming at the problem of the absence of effective coal-rock recognition methods during top coal caving, we propose a new multi-class feature selection approach to detecting full coal falling/30% rock mixture falling/50% rock mixture falling/full rock falling recognition during caving. The method is based on vibration and acoustic sensors fixed under the tail beam of the hydraulic as well as signal processing techniques. Distinctive vibration and acoustic signals generate during caving that depend on the state of coal-rock mixed. Via a contribution threshold $$\lambda_{0}$$ and classification accuracy P, the optimal combination of feature attributes can be automatically and sequentially decided by using the multi-class F-score (MF-Score) feature reduction approach proposed in this paper. The main idea of the MF-Score is to construct the within-class scatter matrix with a membership function based on the aggregation degree within the sample. The support vector machine classifier has been employed to test the proposed algorithm and identified the state of the coal-rock mixed. The contribution threshold is readjusted according to the calculated feature selection criterion J( k) and user requirement, and then a feedback loop is developed to keep the dynamic of the samples. Experimental results show the good generality and effectiveness of this new approach. [ABSTRACT FROM AUTHOR]

Decision tree learning algorithms are known to be unstable, such that small changes in the training data can result in highly different output models. Instability is an important issue in the context of machine learning which is usually overlooked. In this paper, we illustrate and discuss the problem of instability of decision tree induction algorithms and propose a framework to induce more stable decision trees. In the proposed framework, the split test encompasses two advantageous properties: First, it is able to contribute multiple attributes. Second, it has a polylithic structure. The first property alleviates the race between the competing attributes to be installed at an internal node, which is the major cause of instability. The second property has the potential of improving the stability by providing the locality of the effect of the instances on the split test. We illustrate the effectiveness of the proposed framework by providing a complying decision tree learning algorithm and conducting several experiments. We have evaluated the structural stability of the algorithms by employing three measures. The experimental results reveal that the decision trees induced by the proposed framework exhibit great stability and competitive accuracy in comparison with several well-known decision tree learning algorithms. [ABSTRACT FROM AUTHOR]

Posterior capsule opacification (PCO) is the most common complication of cataract surgery, occurring in up to 50% of patients by 2-3 years after the operation [Spalton in Eye 13(Pt 3b):489-492, ]. This paper proposes a new approach for the assessment of PCO digital images. The approach deploys an unsupervised learning technique for clustering image pixels into different regions based on chromatic attributes. The innovative aspect of this paper lies in proposing the number of regions in a clustered image as a measurement tool for assessing the PCO. Experiments using synthetic data confirmed the plausibility of this approach. A series of experiments conducted on real PCO images demonstrated the robustness and stability of the proposed algorithm. Finally, the comparison of our method's assessment with medical expert evaluation reveals a very reasonable concordance. [ABSTRACT FROM AUTHOR]

In this paper an unsupervised on-line deep learning algorithm for action recognition in video sequences is proposed. Deep learning models capable of deriving spatio-temporal data have been proposed in the past with remarkable results, yet, they are mostly restricted to building features from a short window length. The model presented here, on the other hand, considers the entire sample sequence and extracts the description in a frame-by-frame manner. Each computational node of the proposed paradigm forms clusters and computes point representatives, respectively. Subsequently, a first-order transition matrix stores and continuously updates the successive transitions among the clusters. Both the spatial and temporal information are concurrently treated by the Viterbi Algorithm, which maximizes a criterion based upon (a) the temporal transitions and (b) the similarity of the respective input sequence with the cluster representatives. The derived Viterbi path is the node's output, whereas the concatenation of nine vicinal such paths constitute the input to the corresponding upper level node. The engagement of ART and the Viterbi Algorithm in a Deep learning architecture, here, for the first time, leads to a substantially different approach for action recognition. Compared with other deep learning methodologies, in most cases, it is shown to outperform them, in terms of classification accuracy. [ABSTRACT FROM AUTHOR]

In many classification problems, neighbor data labels have inherent sequential relationships. Sequential learning algorithms take benefit of these relationships in order to improve generalization. In this paper, we revise the multi-scale sequential learning approach (MSSL) for applying it in the multi-class case (MMSSL). We introduce the error-correcting output codesframework in the MSSL classifiers and propose a formulation for calculating confidence maps from the margins of the base classifiers. In addition, we propose a MMSSL compression approach which reduces the number of features in the extended data set without a loss in performance. The proposed methods are tested on several databases, showing significant performance improvement compared to classical approaches. [ABSTRACT FROM AUTHOR]

Choosing a suitable classifier for a given dataset is an important part of developing a pattern recognition system. Since a large variety of classification algorithms are proposed in literature, non-experts do not know which method should be used in order to obtain good classification results on their data. Meta-learning tries to address this problem by recommending promising classifiers based on meta-features computed from a given dataset. In this paper, we empirically evaluate five different categories of state-of-the-art meta-features for their suitability in predicting classification accuracies of several widely used classifiers (including Support Vector Machines, Neural Networks, Random Forests, Decision Trees, and Logistic Regression). Based on the evaluation results, we have developed the first open source meta-learning system that is capable of accurately predicting accuracies of target classifiers. The user provides a dataset as input and gets an automatically created high-performance ready-to-use pattern recognition system in a few simple steps. A user study of the system with non-experts showed that the users were able to develop more accurate pattern recognition systems in significantly less development time when using our system as compared to using a state-of-the-art data mining software. [ABSTRACT FROM AUTHOR]

Textual information in images and video frames constitutes a valuable source of high-level semantics for multimedia indexing and retrieval systems. Text detection is the most crucial step in a multimedia text extraction system and although it has been extensively studied the past decade still, it does not exist a generic architecture that would work for artificial and scene text in multimedia content. In this paper we propose a system for text detection of both artificial and scene text in images and video frames. The system is based on a machine learning stage which uses an Random Forest classifier and a highly discriminative feature set produced by using a new texture operator called Multilevel Adaptive Color edge Local Binary Pattern (MACeLBP). MACeLBP describes the spatial distribution of color edges in multiple adaptive levels of contrast. Then, a gradient-based algorithm is applied to achieve distinction among text lines as well as refinement in the localization of the text lines. The whole algorithm is situated in a multiresolution framework to achieve invariance to scale for the detection of text lines. Finally, an optional connected-component step segments text lines into words based on the distances between the resulting components. The experimental results are produced by applying a concise evaluation methodology and prove the superior performance achieved by the proposed text detection system for artificial and scene text in images and video frames. [ABSTRACT FROM AUTHOR]

In this paper, we propose a survey concerning the state of the art of the graph matching problem, conceived as the most important element in the definition of inductive inference engines in graph-based pattern recognition applications. We review both methodological and algorithmic results, focusing on inexact graph matching procedures. We consider different classes of graphs that are roughly differentiated considering the complexity of the defined labels for both vertices and edges. Emphasis will be given to the understanding of the underlying methodological aspects of each identified research branch. A selection of inexact graph matching algorithms is proposed and synthetically described, aiming at explaining some significant instances of each graph matching methodology mainly considered in the technical literature. [ABSTRACT FROM AUTHOR]

When applying machine learning technology to real-world applications, such as visual quality inspection, several practical issues need to be taken care of. One problem is posed by the reality that usually there are multiple human operators doing the inspection, who will inevitably contradict each other for some of the products to be inspected. In this paper an architecture for learning visual quality inspection is proposed which can be trained by multiple human operators, based on trained ensembles of classifiers. Most of the applicable ensemble techniques have however difficulties learning in these circumstances. In order to effectively train the system a novel ensemble framework is proposed as an enhancement of the grading ensemble technique-called active grading. The active grading algorithms are evaluated on data obtained from a real-world industrial system for visual quality inspection of the printing of labels on CDs, which was labelled independently by four different human operators and their supervisor, and compared to the standard grading algorithm and a range of other ensemble (classifier fusion) techniques. [ABSTRACT FROM AUTHOR]

Abstract  A two-class data set is said to be imbalanced when one (minority) class is heavily under-represented with respect to the other (majority) class. In the presence of a significant overlapping, the task of learning from imbalanced data can be a very difficult problem. Additionally, if the overall imbalance ratio is different from local imbalance ratios in overlap regions, the task can become in a major challenge. This paper explains the behaviour of the k-nearest neighbour (k-NN) rule when learning from such a complex scenario. This local model is compared to other machine learning algorithms, attending to how their behaviour depends on a number of data complexity features (global imbalance, size of overlap region, and its local imbalance). As a result, several conclusions useful for classifier design are inferred. [ABSTRACT FROM AUTHOR]