Search

• The greedy AutoAugment is proposed to find the best augmentation policies. • It reduces the exponential growth of the number of possible trials to linear growth. • The algorithm is computationally efficient and can concatenate many sub-policies. • The greedy approach helps the algorithm to increase the accuracy of the network. A major problem in data augmentation is to ensure that the generated new samples cover the search space. This is a challenging problem and requires exploration for data augmentation policies to ensure their effectiveness in covering the search space. In this paper, we propose Greedy AutoAugment as a highly efficient search algorithm to find the best augmentation policies. We use a greedy approach to reduce the exponential growth of the number of possible trials to linear growth. The Greedy Search also helps us to lead the search towards the sub-policies with better results, which eventually helps to increase the accuracy. The proposed method can be used as a reliable addition to the current artifitial neural networks. Our experiments on four datasets (Tiny ImageNet, CIFAR-10, CIFAR-100, and SVHN) show that Greedy AutoAugment provides better accuracy, while using 360 times fewer computational resources. [ABSTRACT FROM AUTHOR]

• A supervised classification based MRF model is proposed for depth map up-sampling. • Low-resolution depth segmentation is used to supervise color image classification. • Classification result is further introduced in the design of MRF energy function. • The designed MRF energy function is optimized with a gradient descent algorithm. • Comparisons with the state-of-the-art show the superiority of the proposed SC-MRF model. In this paper, a Supervised Classification assisted Markov Random Field (SC-MRF) model is proposed for generating high-quality up-sampled depth maps. The proposed model aims to reduce depth bleeding and depth confusion artifacts that can be produced at boundary regions of the up-sampled depth maps. In the proposed model, segmentation of low-resolution (LR) depth map is first used to supervise the classification of corresponding high-resolution (HR) color image. With this supervised classification, not only can the depth edges be retained, but redundant textures in the HR color image can be omitted. The classification result is then introduced into the design of a MRF energy function, and the final up-sampled depth map is obtained by optimizing this energy function with the gradient descent algorithm. For simplicity, classical K -means clustering is adopted to segment the LR depth map into several classes, and a feature-based K -nearest neighbour (K -NN) method is utilized for the supervised classification. With the proposed SC-MRF model, interaction between depths of different classes will be strongly suppressed, meaning depth edges are well preserved. Comparisons with the state-of-the-art demonstrate the strong performance of the proposed method both visually and by quantitative evaluation. [ABSTRACT FROM AUTHOR]

• A new graph-based data classification method is presented. • The method can be applied to both semi-supervised and unsupervised learning tasks. • Some of its properties allow it to perform well with small labeled training sets. • It is unconditionally stable, incorporates class size information and efficient. • Experiments are performed on classification of 3D sensory data such as point clouds. Data classification, where the goal is to divide data into predefined classes, is a fundamental problem in machine learning with many applications, including the classification of 3D sensory data. In this paper, we present a data classification method which can be applied to both semi-supervised and unsupervised learning tasks. The algorithm is derived by unifying complementary region-based and edge-based approaches; a gradient flow of the optimization energy is performed using modified auction dynamics. In addition to being unconditionally stable and efficient, the method is equipped with several properties allowing it to perform accurately even with small labeled training sets, often with considerably fewer labeled training elements compared to competing methods; this is an important advantage due to the scarcity of labeled training data. Some of the properties are: the embedding of data into a weighted similarity graph, the in-depth construction of the weights using, e.g., geometric information, the use of a combination of region-based and edge-based techniques, the incorporation of class size information and integration of random fluctuations. The effectiveness of the method is demonstrated by experiments on classification of 3D point clouds; the algorithm classifies a point cloud of more than a million points in 1–2 min. [ABSTRACT FROM AUTHOR]

Abstract: This paper proposes a novel criterion for estimating the redundancy information of selected feature sets in multi-dimensional pattern classification. An appropriate feature selection process typically maximizes the relevancy of features to each class and minimizes the redundancy of features between selected features. Unlike to the relevancy information that can be measured by mutual information, however, it is difficult to estimate the redundancy information because its dynamic range is varied by the characteristics of features and classes. By utilizing the conceptual diagram of the relationship between candidate features, selected features, and class variables, this paper proposes a new criterion to accurately compute the amount of redundancy. Specifically, the redundancy term is estimated by conditional mutual information between selected and candidate features to each class variable, which does not need a cumbersome normalization process as the conventional algorithm does. The proposed algorithm is implemented into a speech/music discrimination system to evaluate classification performance. Experimental results by varying the number of selected features verify that the proposed method shows higher classification accuracy than conventional algorithms. [ABSTRACT FROM AUTHOR]

For high-dimensional data, the appropriate selection of features has a significant effect on the cost and accuracy of an automated classifier. In this paper, a feature selection technique using genetic algorithms is applied. For classification, crisp and fuzzy k-nearest neighbor (kNN) classifiers are compared. Composite fuzzy classifier architectures are investigated. Experiments are conducted on airborne visible/infrared imaging spectrometer (AVIRIS) data, and the results are evaluated in the paper. [Copyright &y& Elsevier]

Abstract: Semantic Role Labeling (SRL) systems aim at determining the semantic role labels of the arguments of the predicates in natural language text. SRL systems can usually be built to work upon the result of constitient analysis (constituent-based), or dependency parsing (dependency-based). SRL systems can use either classification or sequence labeling as the main processing mechanism. In this paper, we show that a dependency-based SRL system using sequence labeling can achieve state-of-the-art performance when a new structural SVM adapted from the Pegasos algorithm is exploited for performing sequence labeling. [Copyright &y& Elsevier]

Abstract: Ensemble methods have been successfully used as a classification scheme. The reduction of the complexity of this popular learning paradigm motivated the appearance of ensemble pruning algorithms. This paper presents a new ensemble pruning method which highly reduces the complexity of ensemble methods and performs better than complete bagging in terms of classification accuracy. More importantly, it is a very fast algorithm. It consists in ordering all base classifiers with respect to a new criterion which exploits an unsupervised ensemble margin. This method highlights the major influence of low margin instances on the performance of the pruning task and, more generally, the potential of low margin instances for the design of better ensembles. Comparison to both the naive approach of randomly pruning base classifiers and another ordering-based pruning algorithm is carried out in an extensive empirical analysis. [Copyright &y& Elsevier]

Abstract: Object discovery in visual object categorisation (VOC) is the problem of automatically assigning class labels to objects appearing in given images. To achieve state-of-the-art results in this task, a large set of positive and negative training images from publicly available benchmark data sets have been used to train discriminative classification methods. The immediate drawback of these methods is the requirement of a vast amount of labelled data. Therefore, the ultimate challenge for visual object categorisation has been recently exposed: unsupervised object discovery, also called unsupervised VOC (UVOC), where the selection of the number of classes and the assignments of given images to these classes are performed automatically. The problem is very challenging and hitherto only a few methods have been proposed. These methods are based on the popular bag-of-features approach and clustering to automatically form the classes. In this paper, we adopt the self-organising principle and replace clustering with the self-organising map (SOM) algorithm. Our method provides results comparable to the state of the art and its advantages, such as non-sensitivity against codebook histogram normalisation, advocate its usage in unsupervised object discovery. [Copyright &y& Elsevier]

Abstract: In this paper, we present a hypergraph based geometric biclustering (HGBC) algorithm. In a high dimensional space, bicluster patterns to be recognized can be considered to be linear geometrical structures. We can use the Hough transform (HT) to find sub-biclusters which correspond to the linear structures in column-pair spaces. Then a hypergraph model is built to merge the sub-biclusters into larger ones. Experiments on simulated and real biological data show that the HGBC algorithm proposed here can combine the sub-biclusters efficiently and provide more accurate classification results compared with existing biclustering methods. [Copyright &y& Elsevier]

Abstract: The field of pattern recognition is usually subdivided into the statistical and the structural approach. Structural pattern recognition allows one to use powerful and flexible representation formalisms but offers only a limited repertoire of algorithmic tools needed to solve classification and clustering problems. By contrast, the statistical approach is mathematically well founded and offers many tools, but provides a representation formalism that is limited in its power and flexibility. Hence, both subfields are complementary to each other. During the last three decades several efforts have been made towards bridging the gap between structural and statistical pattern recognition in order to profit from the benefits of each approach and eliminate the drawbacks. The present paper reviews some of these attempts made towards the unification of structural and statistical pattern recognition and analyzes the progress that has been achieved. [Copyright &y& Elsevier]

Abstract: Isometric mapping (Isomap) is a popular nonlinear dimensionality reduction technique which has shown high potential in visualization and classification. However, it appears sensitive to noise or scarcity of observations. This inadequacy may hinder its application for the classification of microarray data, in which the expression levels of thousands of genes in a few normal and tumor sample tissues are measured. In this paper we propose a double-bounded tree-connected variant of Isomap, aimed at being more robust to noise and outliers when used for classification and also computationally more efficient. It differs from the original Isomap in the way the neighborhood graph is generated: in the first stage we apply a double-bounding rule that confines the search to at most k nearest neighbors contained within an ε-radius hypersphere; the resulting subgraphs are then joined by computing a minimum spanning tree among the connected components. We therefore achieve a connected graph without unnaturally inflating the values of k and ε. The computational experiences show that the new method performs significantly better in terms of accuracy with respect to Isomap, k-edge-connected Isomap and the direct application of support vector machines to data in the input space, consistently across seven microarray datasets considered in our tests. [Copyright &y& Elsevier]

Abstract: Citrus canker, a bacterial disease of citrus tree leaves, causes significant damage to citrus production worldwide. Effective and fast disease detection methods must be undertaken to minimize the losses of citrus canker infection. In this paper, we present a new approach based on global features and zone-based local features to detect citrus canker from leaf images collected in field which is more difficult than the leaf images captured in labs. Firstly, an improved AdaBoost algorithm is used to select the most significant features of citrus lesions for the segmentation of the lesions from their background. Then a canker lesion descriptor is proposed which combines both color and local texture distribution of canker lesion zones suggested by plant phytopathologists. A two-level hierarchical detection structure is developed to identify canker lesions. Thirdly, we evaluate the proposed method and its comparison with other approaches, and the experimental results show that the proposed approach achieves similar classification accuracy as human experts. [Copyright &y& Elsevier]

Abstract: Discretization of continuous data is one of the important pre-processing tasks in data mining and knowledge discovery. Generally speaking, discretization can lead to improved predictive accuracy of induction algorithms, and the obtained rules are normally shorter and more understandable. In this paper, we present the Class-Attribute Coherence Maximization (CACM) algorithm and the Efficient-CACM algorithm. We have compared the performance of our algorithms with the most relevant discretization algorithm, Fast Class-Attribute Interdependence Maximization (Fast-CAIM) discertization algorithm (). Empirical evaluation of our algorithms and Fast-CAIM on 12 well-known datasets shows that ours generate the superior discretization scheme, which can significantly improve the classification performance of C4.5 and RBF-SVM classifier. As to the execution time of discretization, ours also prove faster than Fast-CAIM algorithm, with the Efficient-CACM algorithm having the shortest execution time. [Copyright &y& Elsevier]

Abstract: Text categorization presents unique challenges to traditional classification methods due to the large number of features inherent in the datasets from real-world applications of text categorization, and a great deal of training samples. In high-dimensional document data, the classes are typically categorized only by subsets of features, which are typically different for the classes of different topics. This paper presents a simple but effective classifier for text categorization using class-dependent projection based method. By projecting onto a set of individual subspaces, the samples belonging to different document classes are separated such that they are easily to be classified. This is achieved by developing a new supervised feature weighting algorithm to learn the optimized subspaces for all the document classes. The experiments carried out on common benchmarking corpuses showed that the proposed method achieved both higher classification accuracy and lower computational costs than some distinguishing classifiers in text categorization, especially for datasets including document categories with overlapping topics. [Copyright &y& Elsevier]

Abstract: Marginal information is of great importance for classification. This paper presents a new nonparametric linear discriminant analysis method named Push–Pull marginal discriminant analysis (PPMDA), which takes full advantage of marginal information. For two-class cases, the idea of this method is to determine projected directions such that the marginal samples of one class are pushed away from the between-class marginal samples as far as possible and simultaneously pulled to the within-class samples as close as possible. This idea can be extended for multi-class cases and give rise to the PPMDA algorithm for feature extraction of multi-class problems. The proposed method is evaluated using the CENPARMI handwritten numeral database, the Extended Yale face database B and the ORL database. Experimental results show the effectiveness of the proposed method and its advantage after performance over the state-of-the-art feature extraction methods. [Copyright &y& Elsevier]

Abstract: Boosting has become one of the state-of-the-art techniques in many supervised learning and semi-supervised learning applications. In this paper, we develop a novel boosting algorithm, MTBoost, for multi-task learning problem. Many previous multi-task learning algorithms can only solve the problem in low or moderate dimensional space. However, the MTBoost algorithm is capable of working for very high dimensional data such as in text mining where the feature number is beyond several 10,000. The experimental results illustrate that the MTBoost algorithm provides significantly better classification performance than supervised single task learning algorithms. Moreover, MTBoost outperforms some other typical multi-task learning methods. [ABSTRACT FROM AUTHOR]

Abstract: Light Detection And Ranging (LIDAR) is an important modality in terrain and land surveying for many environmental, engineering and civil applications. This paper presents the framework for a recently developed unsupervised classification algorithm called Skewness Balancing for object and ground point separation in airborne LIDAR data. The main advantages of the algorithm are threshold-freedom and independence from LIDAR data format and resolution, while preserving object and terrain details. The framework for Skewness Balancing has been built in this contribution with a prediction model in which unknown LIDAR tiles can be categorised as “hilly” or “moderate” terrains. Accuracy assessment of the model is carried out using cross-validation with an overall accuracy of 95%. An extension to the algorithm is developed to address the overclassification issue for hilly terrain. For moderate terrain, the results show that from the classified tiles detached objects (buildings and vegetation) and attached objects (bridges and motorway junctions) are separated from bare earth (ground, roads and yards) which makes Skewness Balancing ideal to be integrated into geographic information system (GIS) software packages. [Copyright &y& Elsevier]

Abstract: Functional data are difficult to manage for most classical statistical techniques, given the very high (or intrinsically infinite) dimensionality. The reason lies in that functional data are functions and most algorithms are designed to work with low dimensional vectors. In this paper we propose a functional analysis technique to obtain finite-dimensional representations of functional data. The key idea is to consider each functional datum as a point in a general function space and then to project these points onto a Reproducing Kernel Hilbert Space with the aid of a support vector machine. We show some theoretical properties of the method and illustrate its performance in some classification examples. [Copyright &y& Elsevier]

Abstract: In this paper we present InstanceRank, a ranking algorithm that reflects the relevance of the instances within a dataset. InstanceRank applies a similar solution to that used by PageRank, the web pages ranking algorithm in the Google search engine. We also present ISR, an instance selection technique that uses InstanceRank. This algorithm chooses the most representative instances from a learning database. Experiments show that ISR algorithm, with InstanceRank as ranking criteria, obtains similar results in accuracy to other instance reduction techniques, noticeably reducing the size of the instance set. [Copyright &y& Elsevier]

Abstract: This paper presents a novel ensemble classifier generation technique RotBoost, which is constructed by combining Rotation Forest and AdaBoost. The experiments conducted with 36 real-world data sets available from the UCI repository, among which a classification tree is adopted as the base learning algorithm, demonstrate that RotBoost can generate ensemble classifiers with significantly lower prediction error than either Rotation Forest or AdaBoost more often than the reverse. Meanwhile, RotBoost is found to perform much better than Bagging and MultiBoost. Through employing the bias and variance decompositions of error to gain more insight of the considered classification methods, RotBoost is seen to simultaneously reduce the bias and variance terms of a single tree and the decrement achieved by it is much greater than that done by the other ensemble methods, which leads RotBoost to perform best among the considered classification procedures. Furthermore, RotBoost has a potential advantage over AdaBoost of suiting parallel execution. [Copyright &y& Elsevier]

Abstract: Various sequence-similarity kernels, the string kernels, have been introduced for use with support vector machines (SVMs) in a discriminative approach to the sequence data classification problems. In these applications, string kernels are asked to be similarity measures between strings. In this paper, we present a new string kernel and its variants suitable to sequence data classification, which are determined by (possibly non-contiguous) matching subsequences with all possible lengths shared by two strings. In these kernels, gaps in subsequences are allowed and the longer subsequences contribute more to the value of kernels. Efficient algorithms of computing the kernels are derived with the techniques of dynamic programming and bit-parallelism. In some cases, the computation of the kernel is linear in the length of the strings. [Copyright &y& Elsevier]

A fuzzy extension of the Rand index [Rand, W.M., 1971. Objective criteria for the evaluation of clustering methods. J. Amer. Statist. Assoc. 846–850] is introduced in this paper. The Rand index is a traditional criterion for assessment and comparison of different results provided by classifiers and clustering algorithms. It is able to measure the quality of different hard partitions of a data set from a classification perspective, including partitions with different numbers of classes or clusters. The original Rand index is extended here by making it able to evaluate a fuzzy partition of a data set – provided by a fuzzy clustering algorithm or a classifier with fuzzy-like outputs – against a reference hard partition that encodes the actual (known) data classes. A theoretical formulation based on formal concepts from the fuzzy set theory is derived and used as a basis for the mathematical interpretation of the Fuzzy Rand Index proposed. The fuzzy counterparts of other (five) related indexes, namely, the Adjusted Rand Index of Hubert and Arabie, the Jaccard coefficient, the Minkowski measure, the Fowlkes–Mallows Index, and the Γ statistics, are also derived from this formulation. [Copyright &y& Elsevier]

Abstract: Combining all features coded by different systems can improve the performance of a classification system. In this paper, a strategy of boosting based feature combination is introduced, where a variant of boosting is proposed for integrating different features. Different from the general boosting, at each round of this variant boosting, some weak classifiers are built on different feature sets, one of which is trained on one feature set. And then these classifiers are combined by weighted voting into a single one as the output classifier of this round. Experiments on classification of three UCI data sets and handwritten digit recognition indicate that this variant of boosting is a promising learning algorithm for feature combination. To some extent, this strategy of feature combination can integrate feature selection, feature communication, and classifier learning in its learning procedure. [Copyright &y& Elsevier]

Abstract: The transductive support vector machine (TSVM) is the transductive inference of the support vector machine. The TSVM utilizes the information carried by the unlabeled samples for classification and acquires better classification performance than the regular support vector machine (SVM). As effective as the TSVM is, it still has obvious deficiency: The number of positive samples must be appointed before training and it is not changed during the training phase. This deficiency is caused by the pair-wise exchanging criterion used in the TSVM. In this paper, we propose a new transductive training algorithm by substituting the pair-wise exchanging criterion with the individually judging and changing criterion. Experimental results show that the new method releases the restriction of the appointment of the number of positive samples beforehand and improves the adaptability of the TSVM. [Copyright &y& Elsevier]

Abstract: Evolutionary algorithms has been recently used for prototype selection showing good results. An important problem that we can find is the scaling up problem that appears evaluating the Evolutionary Prototype Selection algorithms in large size data sets. In this paper, we offer a proposal to solve the drawbacks introduced by the evaluation of large size data sets using evolutionary prototype selection algorithms. In order to do this we have proposed a combination of stratified strategy and CHC as representative evolutionary algorithm model. This study includes a comparison between our proposal and other non-evolutionary prototype selection algorithms combined with the stratified strategy. The results show that stratified evolutionary prototype selection consistently outperforms the non-evolutionary ones, the main advantages being: better instance reduction rates, higher classification accuracy and reduction in resources consumption. [Copyright &y& Elsevier]

Aggregating outputs of multiple classifiers into a committee decision is one of the most important techniques for improving classification accuracy. The issue of selecting an optimal subset of relevant features plays also an important role in successful design of a pattern recognition system. In this paper, we present a neural network based approach for identifying salient features for classification in neural network committees. Feature selection is based on two criteria, namely the reaction of the cross-validation data set classification error due to the removal of the individual features and the diversity of neural networks comprising the committee. The algorithm developed removed a large number of features from the original data sets without reducing the classification accuracy of the committees. The accuracy of the committees utilizing the reduced feature sets was higher than those exploiting all the original features. [Copyright &y& Elsevier]

The S–K algorithm (Schlesinger–Kozinec algorithm) and the modified kernel technique due to Friess et al. have been recently combined to solve SVM with L2 cost function. In this paper, we generalize S–K algorithm to be applied for soft convex hulls. As a result, our algorithm can solve ν-SVM based on L1 cost function. Simple in nature, our soft algorithm is essentially a algorithm for finding the ε-optimal nearest points between two soft convex hulls. As only the vertexes of the hard convex hulls are used, the obvious superiority of our algorithm is that it has almost the same computational cost as that of the hard S–K algorithm. The theoretical analysis and some experiments demonstrate the performance of our algorithm. [Copyright &y& Elsevier]

In this paper, an efficient hierarchical clustering algorithm, suitable for large data sets is proposed for effective clustering and prototype selection for pattern classification. It is another simple and efficient technique which uses incremental clustering principles to generate a hierarchical structure for finding the subgroups/subclusters within each cluster. As an example, a two level clustering algorithm––‘Leaders–Subleaders’, an extension of the leader algorithm is presented. Classification accuracy (CA) obtained using the representatives generated by the Leaders–Subleaders method is found to be better than that of using leaders as representatives. Even if more number of prototypes are generated, classification time is less as only a part of the hierarchical structure is searched. [Copyright &y& Elsevier]

The paper reports on the robust pattern classification of experimental data using a combined approach of symbolization followed by support vector machine (SVM) classification. Symbolization of data removes unwanted features such as noise whereas SVM provides the classification. The SVM parameters are tuned on-line using a genetic–quasi-Newton algorithm. Benchmark examples illustrate the proposed approach. [Copyright &y& Elsevier]

In this paper, we propose a method of content-based image classification using a neural network. The images for classification are object images that can be divided into foreground and background. To deal with the object images efficiently, in the preprocessing step we extract the object region using a region segmentation technique. Features for the classification are shape-based texture features extracted from wavelet-transformed images. The neural network classifier is constructed for the features using the back-propagation learning algorithm. Among the various texture features, the diagonal moment was the most effective. A test with 300 training data and 300 test data composed of 10 images from each of 30 classes shows classification rates of 81.7% and 76.7% correct, respectively. [Copyright &y& Elsevier]

Feature selection plays an important role in pattern classification. In this paper, we present an improved branch and bound algorithm for optimal feature subset selection. This algorithm searches for an optimal solution in a large solution tree in an efficient manner by cutting unnecessary paths which are guaranteed not to contain the optimal solution. Our experimental results demonstrate the effectiveness of the new algorithm. [Copyright &y& Elsevier]

The performance of most practical classifiers improves when correlated or irrelevant features are removed. Machine based classification is thus often preceded by subset selection––a procedure which identifies relevant features of a high dimensional data set. At present, the most widely used subset selection technique is the so-called “wrapper” approach in which a search algorithm is used to identify candidate subsets and the actual classifier is used as a “black box” to evaluate the fitness of the subset. Fitness evaluation of the subset however requires cross-validation or other resampling based procedure for error estimation necessitating the construction of a large number of classifiers for each subset. This significant computational burden makes the wrapper approach impractical when a large number of features are present.In this paper, we present an approach to subset selection based on a novel definition of the classifiability of a given data. The classifiability measure we propose characterizes the relative ease with which some labeled data can be classified. We use this definition of classifiability to systematically add the feature which leads to the most increase in classifiability. The proposed approach does not require the construction of classifiers at each step and therefore does not suffer from as high a computational burden as a wrapper approach. Our results over several different data sets indicate that the results obtained are at least as good as that obtained with the wrapper approach. [Copyright &y& Elsevier]