Search

Highlights • This paper introduces a novel technique to extract a sparse feature vector from extreme low resolution face images. This feature vector enables us to synthesize a high-resolution face image with magnification factors up to 16x from a single input face. We show how our method is robust to noise and handles real-world low-resolution faces. • The significance of this paper lies in the fact that despite its straightforward mathematical foundations (simple subspace modeling followed by a sparse feature extraction step), it yields reconstruction results that comprehensively exceed state of the art and more convoluted methods (such as deep leaning methods SRCNN and SRGAN). Moreover, our method only requires a single input face to perform super resolution. Abstract Automatic face recognition in the wild still suffers from low-quality, low resolution, noisy, and occluded input images that can severely impact identification accuracy. In this paper, we present a novel technique to enhance the quality of such extreme low-resolution face images beyond the current state of the art. We model the correlation between high and low resolution faces in a multi-resolution pyramid and show that we can recover the original structure of an un-seen extreme low-resolution face image. By exploiting domain knowledge of the structure of the input signal and using sparse recovery optimization algorithms, we can recover a consistent sparse representation of the extreme low-resolution signal. The proposed super-resolution method is robust to noise and face alignment, and can handle extreme low-resolution faces up to 16x magnification factor with just 7 pixels between the eyes. Moreover, the formulation of the proposed algorithm allows for simultaneous occlusion removal capability, a desirable property that other super-resolution algorithms do not possess, to the best of our knowledge. Most importantly, we show that our method generalizes on real-world low-quality surveillance images, showing the potentially big impact this can have in a real-world scenario. [ABSTRACT FROM AUTHOR]

Highlights • As far as we know, we are the first to apply CRC on domain adaptation problem. This paper proposed a jointly discriminative projection and dictionary learning method (JD2-CRC). The projection matrices and dictionary are learned according to the classification rule of CRC for improving the discriminability of collaborative representations. • Different to traditional optimization algorithm where a ratio trace problem is solved, we exploit the relationship of optimal variables, and optimize the trace ratio problem directly by a new trace ratio optimization method which is efficient and effective. • We conduct several experiments to verify the performances of the proposed algorithm. The experimental results show that the proposed optimization method is effective, and JD2-CRC achieves better performance than other state-of-the-art methods in domain adaptation problem. Abstract In recent years, collaborative representation-based classification (CRC) methods have shown impressive performance in many recognition tasks. However, when the training data have different distributions with the testing data, the performance of CRC will be degraded significantly. On the other hand, concatenating training data from different sources as a single data set will affect the performance of CRC, as the shift exists between the different source domains. To address these problems, in this paper, we propose a Jointly Discriminative projection and Dictionary learning for domain adaptive Collaborative Representation-based Classification method (JD2-CRC). As the distributions of different source domains may be dissimilar, the data from all domains are projected into a common feature subspace where the latent shared structures can be found. Then a compact dictionary is learned to represent the projected data well. To find the most suitable projection matrices and dictionary for CRC, we design the objective function of JD2-CRC,according to the classification rule of CRC in feature subspace, which minimizes the ratio of within-class reconstruction errors over between-class reconstruction errors. Different to traditional optimization methods, an effective optimization procedure is presented based on gradient descent. Thus, the obtained collaborative representations have a better discriminability and suit the classification rule of CRC well. The experimental results demonstrate that the proposed method can achieve superior performance against other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Abstract: -Means algorithm and its variations are known to be fast clustering algorithms. However, they are sensitive to the choice of starting points and inefficient for solving clustering problems in large data sets. Recently, a new version of the -means algorithm, the global -means algorithm has been developed. It is an incremental algorithm that dynamically adds one cluster center at a time and uses each data point as a candidate for the -th cluster center. Results of numerical experiments show that the global -means algorithm considerably outperforms the -means algorithms. In this paper, a new version of the global -means algorithm is proposed. A starting point for the -th cluster center in this algorithm is computed by minimizing an auxiliary cluster function. Results of numerical experiments on 14 data sets demonstrate the superiority of the new algorithm, however, it requires more computational time than the global -means algorithm. [Copyright &y& Elsevier]

Multi-label classification associates an unseen instance with multiple relevant labels. In recent years, a variety of methods have been proposed to handle the multi-label problems. Classifier chains is one of the most popular multi-label methods because of its efficiency and simplicity. In this paper, we consider to optimize classifier chains from the viewpoint of conditional likelihood maximization. In the proposed unified framework, classifier chains can be optimized in either or both of two aspects: label correlation modeling and multi-label feature selection. In this paper we show that previous classifier chains algorithms are specified in the unified framework. In addition, previous information theoretic multi-label feature selection algorithms are specified with different assumptions on the feature and label spaces. Based on these analyses, we propose a novel multi-label method, k -dependence classifier chains with label-specific features, and demonstrate the effectiveness of the method. [ABSTRACT FROM AUTHOR]

Highlights • Groupwise image registration by sequentially registering pairs of images is error-prone in case of highly unreliable pairwise registrations. • Hough voting spaces enable probabilistic pairwise registration estimations in order to identify the joint max-likelihood registration of all images. • Framework for probabilistic groupwise registration process based on feature correspondences. • Unlike previous groupwise registration work, the proposed method is able to jointly register images with arbitrary translational and rotational differences. • Outperforms state-of-the-art for highly challenging registration of aerial WW2 images to present-day ortho-photo maps. Abstract In this paper, we address the registration of historical WWII images to present-day ortho-photo maps for the purpose of geolocalization. Due to the challenging nature of this problem, we propose to register the images jointly as a group rather than in a step-by-step manner. To this end, we exploit Hough voting spaces as pairwise registration estimators and show how they can be integrated into a probabilistic groupwise registration framework that can be efficiently optimized. The feature-based nature of our registration framework allows to register images with a-priori unknown translational and rotational relations, and is also able to handle scale changes of up to 30% in our test data due to a final geometrically guided matching step. The superiority of the proposed method over existing pairwise and groupwise registration methods is demonstrated on eight highly challenging sets of historical images with corresponding ortho-photo maps. [ABSTRACT FROM AUTHOR]

Highlights • Sampson distance is revisited and its geometric meaning is exhibited. • A new geometric distance between a point and a conic, called Polar-N-Direction distance, is given. It can be adapted to a projective transformation because it is computed along the normal direction of the polar line of the point, making conic fitting more robust and accurate. • Polar-N-Direction distance is represented explicitly and analytically. This geometric distance based fitting is greatly easy to be implemented. • A new cost function based on Polar-N-Direction distance is constructed. The conic fitting optimization by minimizing this cost function is very efficient. Abstract Fitting conics from images is a preliminary step for its plentiful applications. It is a common sense that geometric distance based fitting methods are better than algebraic distance based ones. However, for a long time, there has not been a geometric distance between a point and a general conic that allows easy computation and achieves high accuracy simultaneously. Though Sampson distance is widely accepted, it is only a first-order approximation. For other geometric distances, the computations are too complex to be popular in practice. In this paper, we derive a new geometric distance between a point and a general conic, called Polar-N-Direction distance. The distance can be adapted to a projective transformation because it is computed along the normal direction of the polar line of the point, making conic fitting more robust. Moreover, Polar-N-Direction distance is accurate and simultaneously still analytical in an explicit representation, which is quite easy to be implemented. Then, based on the distance, a new cost function is constructed. The conic fitting optimization by minimizing this cost function has all the merits of the geometric distance based methods and simultaneously avoids their limitations. Experiments show that the conic fitting method is greatly efficient. [ABSTRACT FROM AUTHOR]

Highlights • We present an integer linear programming (ILP) model for the categorical clustering. • A global heuristic is presented for solving the ILP of medium and large size. • Experiments show that the proposed method is superior to the existing approaches. • New best results are provided for the UCI benchmark datasets. Abstract The conventional k -modes algorithm and its variants have been extensively used for categorical data clustering. However, these algorithms have some drawbacks, e.g., they can be trapped into local optima and sensitive to initial clusters/modes. Our numerical experiments even showed that the k -modes algorithm could not identify the optimal clustering results for some special datasets regardless the selection of the initial centers. In this paper, we developed an integer linear programming (ILP) approach for the k -modes clustering, which is independent to the initial solution and can obtain directly the optimal results for small-sized datasets. We also developed a heuristic algorithm that implements iterative partial optimization in the ILP approach based on a framework of variable neighborhood search, known as IPO-ILP-VNS, to search for near-optimal results of medium and large sized datasets with controlled computing time. Experiments on 38 datasets, including 27 synthesized small datasets and 11 known benchmark datasets from the UCI site were carried out to test the proposed ILP approach and the IPO-ILP-VNS algorithm. The experimental results outperformed the conventional and other existing enhanced k -modes algorithms in literature, updated 9 of the UCI benchmark datasets with new and improved results. [ABSTRACT FROM AUTHOR]

Highlights • We proposed a novel ECOC algorithm for multiclass microarray data classification based on the data complexity theory. • Various data complexity measures are deployed to detect the intrinsic characteristics of microarray data sets, so as to produce diverse coding matrices. • A new data complexity measure, named as C1, is designed to evaluate data distribution. It benefits the optimization process of our class partition. • The proposed ECOC algorithm performs more stably in most multiclass microarray data sets compared with other popular ECOC algorithms. Abstract Nowadays, a lot of new classification and clustering techniques have been proposed for microarray data analysis. However, the multiclass microarray data classification is still regarded as a tough task because of the small sample size problem and the class imbalance problem. In this paper, we propose a novel error correcting output code (ECOC) algorithm for the classification of multiclass microarray data based on the data complexity (DC) theory. In this algorithm, an ECOC coding matrix is generated based on a hierarchical partition of the class space with the aim of Minimizing Data Complexity (named as ECOC-MDC). As the partition process can be mapped as a binary tree, a compact ensemble with high discrimination power is produced. The performance of ECOC-MDC is compared with some state-of-art ECOC algorithms on six multiclass microarray data sets, and it is found that the proposed algorithm can obtain better results in most cases. The correlation between DC measures and the dichotomizers' performances is checked, and the observations confirm that high complexity in data usually leads to high error rates of the connected dichotomizers. But the error correcting mechanism in the ECOC framework can effectively improve our algorithm's generalization ability. In short, ECOC-MDC can produce a compact ensemble system with high error correction capability through the application of diverse DC measures. Our Matlab code is available at: github.com/MLDMXM2017/ECOC-MDC. [ABSTRACT FROM AUTHOR]

Abstract This paper proposes a novel regularizer named Structured Decorrelation Constraint, to address both the generalization and optimization of deep neural networks, including multiple-layer perceptrons and convolutional neural networks. Our proposed regularizer reduces overfitting by breaking the co-adaptions between the neurons with an explicit penalty. As a result, the network is capable of learning non-redundant representations. Meanwhile, the proposed regularizer encourages the networks to learn structured high-level features to aid the networks' optimization during training. To this end, neurons are constrained to behave obeying a group prior. Our regularizer applies to various types of layers, including fully connected layers, convolutional layers and normalization layers. The loss of our regularizer can be directly minimized along with the network's classification loss by stochastic gradient descent. Experiments show that the proposed regularizer obviously relieves the overfitting problem of the existing deep networks. It yields much better performance on extensive datasets than the conventional regularizers like Dropout. [ABSTRACT FROM AUTHOR]

In this paper, a novel robust one-class support vector machine (OCSVM) based on the rescaled hinge loss function is proposed to enhance the robustness of the conventional OCSVM against outliers. The optimization problem of the proposed robust OCSVM can be iteratively solved by the half-quadratic optimization technique. Compared to OCSVM, robust OCSVM may achieve higher generalization performance from the theoretical analysis. Moreover, the robustness of robust OCSVM against outliers is explained from the weighted viewpoint. Experimental results on the synthetic and benchmark data sets demonstrate that the proposed robust OCSVM is superior to the conventional OCSVM and the other two related approaches. [ABSTRACT FROM AUTHOR]

Laplacian support vector machine (SVM) for semi-supervised classification has attracted much attention in recent years. As an extension to improve the computational speed, Laplacian twin parametric-margin SVM (LTPSVM) has shown outstanding performance. However, it is still challenging to handle large-scale data. To address this issue, a safe sample screening rule (SSSR) for LTPSVM is proposed in this paper. It could significantly reduce the computational cost. Our proposed SSSR removes most redundant samples, and reduces the scale of optimization problems without sacrificing the optimal accuracy. The most important advantage of SSSR is the safety, i.e., the solutions are exactly the same as the original ones. Numerical experiments on both a synthetical data set and 14 real world data sets have verified the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

The aim of this paper is to design an algorithm based on nonsmooth optimization techniques to solve the minimum sum-of-squares clustering problems in very large data sets. First, the clustering problem is formulated as a nonsmooth optimization problem. Then the limited memory bundle method [Haarala et al., 2007] is modified and combined with an incremental approach to design a new clustering algorithm. The algorithm is evaluated using real world data sets with both the large number of attributes and the large number of data points. It is also compared with some other optimization based clustering algorithms. The numerical results demonstrate the efficiency of the proposed algorithm for clustering in very large data sets. [ABSTRACT FROM AUTHOR]

Low-textured image stitching remains a challenging problem. It is difficult to achieve good alignment of images and it is easy to break images structures are often broken due to insufficient and unreliable point correspondences. Moreover, because of the viewpoint variations between multiple images, the stitched images suffer from projective distortions. To solve these problems, this paper presents a line-guided local warping method with a global similarity constraint for image stitching. Line features which serve well for geometric descriptions and scene constraints, are employed to guide image stitching accurately. On one hand, the line features are integrated into a local warping model through a designed weight function. On the other hand, line features are adopted to impose strong geometric constraints, including line correspondence and line colinearity, to improve the stitching performance through mesh optimization. To mitigate projective distortions, we adopt a global similarity constraint, which is integrated with the projective warps via a designed weight strategy. This constraint causes the final warp to slowly change from a projective to a similarity transformation across the image. Finally, the images undergo a two-stage alignment scheme that provides accurate alignment and reduces projective distortion. We evaluate our method on a series of images and compare it with several other methods. The experimental results demonstrate that the proposed method provides a convincing stitching performance and that it outperforms other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This paper aims to propose a new feature and intensity-based image registration method. The proposed approach is based on the block matching algorithm (Ourselin et al., 2000 [1] ): a displacement field is locally computed by matching spatially invariant intensity sub-blocks of the images before performing an optimization algorithm from this vector field to estimate the transformation. Our approach proposes a new way to calculate the displacement field by matching spatially variant sub-blocks of the images, called General Adaptive Neighborhoods (GANs) (Debayle and Pinoli, 2006 [2] ). These neighborhoods are adaptive with respect to both the intensities and the spatial structures of the image. They represent the patterns within the grayscale images. This paper also presents a consistent shape metric used to match the GANs. The performed qualitative and quantitative experiments show that the proposed GAN matching method provides accurate displacement fields enabling us to perform image rigid registration, even for data from different modalities, that outperforms the classical block matching algorithm with respect to robustness and accuracy criteria. [ABSTRACT FROM AUTHOR]

• A self-supervised semi-supervised NMF method is proposed for data clustering. • We define an effective semi-supervised NMF model with attractive and repulsive interactions. • An ensemble of semi-supervised NMFs is designed to model a self-supervised one. • The self-supervised model generates pseudo supervisory signals to boost semi-supervised learning. • Experimental results demonstrate the effectiveness of the proposed method in semi-supervised clustering tasks. Semi-supervised nonnegative matrix factorization exploits the strengths of matrix factorization in successfully learning part-based representation and is also able to achieve high learning performance when facing a scarcity of labeled data and a large amount of unlabeled data. Its major challenge lies in how to learn more discriminative representations from limited labeled data. Furthermore, self-supervised learning has been proven very effective at learning representations from unlabeled data in various learning tasks. Recent research works focus on utilizing the capacity of self-supervised learning to enhance semi-supervised learning. In this paper, we design an effective Self-Supervised Semi-Supervised Nonnegative Matrix Factorization (S4NMF) in a semi-supervised clustering setting. The S4NMF directly extracts a consensus result from ensembled NMFs with similarity and dissimilarity regularizations. In an iterative process, this self-supervisory information will be fed back to the proposed model to boost semi-supervised learning and form more distinct clusters. The proposed iterative algorithm is used to solve the given problem, which is defined as an optimization problem with a well-formulated objective function. In addition, the theoretical and empirical analyses investigate the convergence of the proposed optimization algorithm. To demonstrate the effectiveness of the proposed model in semi-supervised clustering, we conduct extensive experiments on standard benchmark datasets. The source code for reproducing our results can be found at https://github.com/ChavoshiNejad/S4NMF. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a method for improved ensemble learning, by treating the optimization of an ensemble of classifiers as a compressed sensing problem. Ensemble learning methods improve the performance of a learned predictor by integrating a weighted combination of multiple predictive models. Ideally, the number of models needed in the ensemble should be minimized, while optimizing the weights associated with each included model. We solve this problem by treating it as an example of the compressed sensing problem, in which a sparse solution must be reconstructed from an under-determined linear system. Compressed sensing techniques are then employed to find an ensemble which is both small and effective. An additional contribution of this paper, is to present a new performance evaluation method (a new pairwise diversity measurement) called the roulette-wheel kappa-error. This method takes into account the different weightings of the classifiers, and also reduces the total number of pairs of classifiers needed in the kappa-error diagram, by selecting pairs through a roulette-wheel selection method according to the weightings of the classifiers. This approach can greatly improve the clarity and informativeness of the kappa-error diagram, especially when the number of classifiers in the ensemble is large. We use 25 different public data sets to evaluate and compare the performance of compressed sensing ensembles using four different sparse reconstruction algorithms, combined with two different classifier learning algorithms and two different training data manipulation techniques. We also give the comparison experiments of our method against another five state-of-the-art pruning methods. These experiments show that our method produces comparable or better accuracy, while being significantly faster than the compared methods. [Copyright &y& Elsevier]

The performance of conventional interactive image segmentation methods is strongly affected by seed quantity and position, and it is difficult for them to maintain global data coherence due to the bias that is caused by limited interactions. Furthermore, the pixel-level relationships in these methods are too local to capture long-range connectivity cues, which often causes them to obtain under-segmented results. To solve these problems, this paper proposes an interactive segmentation method that is based on likelihood diffusion and perceptual learning. The diffusive likelihood strategy is proposed for accurately estimating the prior label probability from limited user inputs. Superpixel-level grouping cues are utilized to enforce continuity during the segmentation process. The geometrical adjacency and long-range grouping cues are fused in the proposed framework to ensure that the segmentation results maintain proximity and continuity. The final results can be obtained by applying a joint optimization technique to solve a pair of sub-module functions. Experiments on the Berkeley segmentation data set and the Microsoft GrabCut database demonstrate that the proposed method outperforms state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In this paper, we present a new cross-modal discrete hashing (CMDH) approach to learn compact binary codes for cross-modal multimedia search. Unlike most existing cross-modal hashing methods which usually relax the optimization objective function to obtain hash codes, we develop a discrete optimization framework to jointly learn binary codes and a series of hash functions for each modality, so that the performance drop due to the inferior optimization techniques can be avoided. Specifically, we present two cross-modal hashing algorithms called CMDH-linear and CMDH-kernel under the proposed framework, which performs linear and non-linear mappings to learn binary codes, respectively. Different from existing cross-modal hashing methods which maximize the corrections of hash codes from different modalities, our CMDH learns a set of shared binary codes for samples captured from different modalities, so that the modality gap can be effectively removed in cross-modal multimedia retrieval. To further improve the flexibility of our approach for different scenarios, we extend CMDH to unsupervised CMDH (unCMDH) and discrete multi-modal hashing (MMDH), which learns hash codes for training data without label information and with multi-modal labelled data. Experimental results on three benchmark datasets clearly show that our methods achieve competitive results with the state-of-the-arts. [ABSTRACT FROM AUTHOR]

Nowadays, there is an increasing demand for machine learning techniques which can deal with problems where the instances are produced as a stream or in real time. In these scenarios, online learning is able to learn a model from data that comes continuously. The adaptability, efficiency and scalability of online learning techniques have been gaining interest last years with the increasing amount of data generated every day. In this paper, we propose a novel binary classification approach based on nonlinear mapping functions under an online learning framework. The non-convex optimization problem that arises is split into three different convex problems that are solved by means of Passive-Aggressive Online Learning. We evaluate both the adaptability and generalization of our model through several experiments comparing with the state of the art techniques. We improve significantly the results in several datasets widely used previously by the online learning community. [ABSTRACT FROM AUTHOR]

In this paper, we leverage the manifold structure of visual data in order to improve performance in general optimization problems subject to linear constraints. As the main theoretical result, we show that manifold constraints can be transferred from the data to the optimized variables if these are linearly correlated. We also show that the resulting optimization problem can be solved with an efficient alternating direction method of multipliers that can consistently integrate the manifold constraints during the optimization process. This leads to a simplification of the approach, which instead of directly optimizing on the manifold, we can iteratively recast the problem as the projection over the manifold via an embedding method. The proposed method is extremely versatile since it can be applied to different problems including Kernel Ridge Regression (KRR) and sparse coding which have numerous applications in machine learning and computer vision. In particular, we apply the methods to different problems such as tracking, object recognition and categorization showing a consistent increase of performance with respect to the state of the art. [ABSTRACT FROM AUTHOR]

Person re-identification (Re-ID) usually suffers from noisy samples with background clutter and mutual occlusion, which makes it extremely difficult to distinguish different individuals across the disjoint camera views. In this paper, we propose a novel deep self-paced learning (DSPL) algorithm to alleviate this problem, in which we apply a self-paced constraint and symmetric regularization to help the relative distance metric training the deep neural network, so as to learn the stable and discriminative features for person Re-ID. Firstly, we propose a soft polynomial regularizer term which can derive the adaptive weights to samples based on both the training loss and model age. As a result, the high-confidence fidelity samples will be emphasized and the low-confidence noisy samples will be suppressed at early stage of the whole training process. Such a learning regime is naturally implemented under a self-paced learning (SPL) framework, in which samples weights are adaptively updated based on both model age and sample loss using an alternative optimization method. Secondly, we introduce a symmetric regularizer term to revise the asymmetric gradient back-propagation derived by the relative distance metric, so as to simultaneously minimize the intra-class distance and maximize the inter-class distance in each triplet unit. Finally, we build a part-based deep neural network, in which the features of different body parts are first discriminately learned in the lower convolutional layers and then fused in the higher fully connected layers. Experiments on several benchmark datasets have demonstrated the superior performance of our method as compared with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

While face recognition has been intensively studied in the literature, there are only a few attempts on visual recognition of multiple faces simultaneously in videos, which has potential applications in practical video surveillance. In this paper, we address the problem of visual recognition and tracking of multiple faces in real-world videos involving large pose variation and occlusion. Instead of recognizing individual face independently, we introduce the constraints of inter-frame temporal smoothness and within-frame identity exclusivity on multiple faces in videos, and model the tasks of multiple face recognition (MFR) and multiple face tracking (MFT) jointly in an alternative optimization framework. We show this joint formulation for two different tasks leads to significantly improved MFR accuracy. Specifically, as appearance matching for face instances over consecutive frames plays a critical role in MFT, we propose an identity-specific metric learning method with a part-based object representation to learn a localized transformation for each face subject in an online manner, under which face instances of the same subject over consecutive frames are pulled as close as possible, while those of different subjects are pushed far away. Empirically, we evaluate our method on several MFR sequences against baselines, and the results demonstrate that our method can achieve improved accuracy performance in various challenging recognition scenarios. [ABSTRACT FROM AUTHOR]

Despite the great advances in face-related works in recent years, face recognition across age remains a challenging problem. The traditional approaches to this problem usually include two basic steps: feature extraction and the application of a distance metric, sometimes common space projection is also involved. On the one hand, handling these steps separately ignores the interactions of these components, and on the other hand, the fixed-distance threshold of measurement affects the model’s robustness. In this paper, we present a novel distance metric optimization driven learning approach that integrates these traditional steps via a deep convolutional neural network, which learns feature representations and the decision function in an end-to-end way. Given the labelled training images, we first generate a large number of pairs with a certain proportion of matched and unmatched pairs. For matched pairs, we try to select as many different age instances as possible for each person to learn the identification information that is not affected by age. Then, taking these pairs as input, we aim to enlarge the differences between the unmatched pairs while reducing the variations between the matched pairs, and we update the model parameters by using the mini-batch stochastic gradient descent (SGD) algorithm. Specifically, the distance matrix is used as the top fully connected layer, and the bottom layers representing the image features are integrated with it seamlessly. Thus, the image features and the distance metric can be optimized simultaneously by backward propagation. In particular, we introduce several training strategies to reduce the computational cost and overcome insufficient memory capacity. We evaluate our method on three tasks: age-invariant face identification on the MORPH database, age-invariant face retrieval on the CACD database and age-invariant face verification on CACD-VS database. The experimental results demonstrate the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

Most existing approaches address multi-view subspace clustering problem by constructing the affinity matrix on each view separately and afterwards propose how to extend spectral clustering algorithm to handle multi-view data. This paper presents an approach to multi-view subspace clustering that learns a joint subspace representation by constructing affinity matrix shared among all views. Relying on the importance of both low-rank and sparsity constraints in the construction of the affinity matrix, we introduce the objective that balances between the agreement across different views, while at the same time encourages sparsity and low-rankness of the solution. Related low-rank and sparsity constrained optimization problem is for each view solved using the alternating direction method of multipliers. Furthermore, we extend our approach to cluster data drawn from nonlinear subspaces by solving the corresponding problem in a reproducing kernel Hilbert space. The proposed algorithm outperforms state-of-the-art multi-view subspace clustering algorithms on one synthetic and four real-world datasets. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel incremental subspace and probability mask constrained sparse representation model in Smart and Autonomous Systems (SAS). In contrast to traditional sparse representation based tracking methods, the proposed model uses the trivial templates to model both reconstruction errors caused by sparse representation and the Eigen subspace representation. Besides, to alleviate the fact that the trivial templates can be activated to represent any image patch, we further propose to constrain the trivial templates with a probability mask. A unified objective function is proposed and a customized APG method is developed to effectively solve the optimization problem. In addition, a robust observation likelihood metric is proposed and a Bayes rule based color histogram model is proposed to construct the probability mask. Numerous qualitative and quantitative evaluations demonstrate that our tracker outperforms the state-of-the-art trackers in a wide range of tracking scenarios. [ABSTRACT FROM AUTHOR]

This paper presents a fast and accurate affine canonicalization method for planar shapes. This method improves on previous ones based on iterative optimization that produce multiple canonical versions. Canonicalization provides a common reference frame for shape comparison without the loss of discrimination ability often caused by invariant features. It also gives for free the alignment transformation between any pair of shapes. The proposed method is based on the properties of the joint angular distribution of marginal skewness and kurtosis, the so-called SK signature , which can be efficiently computed in closed form from the raw image moments. The experiments demonstrate that the method is robust to the non-affine distortions caused by natural perspective image conditions. Thus, it can be used as an automatic preprocessing step to add affine invariance in statistical pattern recognition applications. [ABSTRACT FROM AUTHOR]

This paper deals with Weakly Supervised Learning (WSL), i.e. performing image classification by leveraging local information with models trained from global image labels. We propose a new WSL method which incorporates gaze features collected by an eye-tracker to guide the region selection policy. Our approach presents two appealing advantages: gaze features are cheap to collect, e.g. one order of magnitude faster than bounding boxes, and our method only requires gaze features during training, while being gaze free at test time. For this purpose, the training objective is enriched with a gaze loss, from which we derive a concave-convex upper bound, leading to an off-the-shelf CCCP optimization scheme.  Extensive experiments are conducted to validate the effectiveness of the approach for WSL image classification on two public datasets with gaze annotation, i.e. PASCAL VOC 2012 action and POET. In addition, we publicly release a new food-related dataset, the Gaze-based UPMC Food dataset (UPMC-G20), which covers 20 food categories and 2,000 images. This dataset intends to promote the research in the food-related computer vision community. [ABSTRACT FROM AUTHOR]

Non-rigid structure-from-motion (NRSfM) has so far been mostly studied for recovering 3D structure of a single non-rigid/deforming object. To handle the real world challenging multiple deforming objects scenarios, existing methods either pre-segment different objects in the scene or treat multiple non-rigid objects as a whole to obtain the 3D non-rigid reconstruction. However, these methods fail to exploit the inherent structure in the problem as the solution of segmentation and the solution of reconstruction could not benefit each other. In this paper, we propose a unified framework to jointly segment and reconstruct multiple non-rigid objects. To compactly represent complex multi-body non-rigid scenes, we propose to exploit the structure of the scenes along both temporal direction and spatial direction, thus achieving a spatio-temporal representation. Specifically, we represent the 3D non-rigid deformations as lying in a union of subspaces along the temporal direction and represent the 3D trajectories as lying in the union of subspaces along the spatial direction. This spatio-temporal representation not only provides competitive 3D reconstruction but also outputs robust segmentation of multiple non-rigid objects. The resultant optimization problem is solved efficiently using the Alternating Direction Method of Multipliers (ADMM). Extensive experimental results on both synthetic and real multi-body NRSfM datasets demonstrate the superior performance of our proposed framework compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In this paper, we consider the problem of approximate nearest neighbor (ANN) retrieval with the method of sparse coding, which is a promising tool of discovering compact representation of high-dimensional data. A new study, exploiting the indices of the active set of sparse coded data as retrieval code, exhibits an appealing ANN route. Here our work aims to enhance the method via considering its shortages of the local structure of the data. Our primary innovation is two-fold: We introduce the graph Laplacian regularization to preserve the local structure of the original data into reduced space, which is indeed beneficial to ANN. And we impose non-negativity constraints such that the retrieval code can more effectively reflect the neighborhood relation, thereby cutting down on unnecessary hash collision. To this end, we learn an incoherent dictionary with both rules via a novel formulation of sparse coding. The resulting optimization problem can be provided with an available solution by the widely used iterative scheme, where we resort to the feature-sign search algorithm in the sparse coding step and exploit the method that uses a Lagrange dual for dictionary learning step. Experimental results on publicly available image data sets manifest that the rules are positive to promote the classification and ANN accuracies. Compared with several state-of-the-art ANN techniques, our methods can achieve an interesting improvement in retrieval accuracy. [ABSTRACT FROM AUTHOR]

In this paper we put under the same umbrella several well known results and establish several new results related to the shape based tools which treat the objects ellipticity issues. We start with a derivation of an explicit and closed formula for the computation of the ellipticity measures, from an infinite family, introduced recently. The new formula enables a fast computation of such measures, since it does not require any optimizing procedure for the computation, as it was the case before. In addition, the established formula enables an easy theoretical manipulation. As a result, we have discovered new shape features, as they are: (i) The average shape ellipticity measure , which might be interpreted as an average value of the estimated similarities between the shape considered and all the ellipses whose axes length ratio belongs to a certain, predefined, interval; (ii) The maximal shape ellipticity measure , which might be understood as the maximal possible similarity estimate between a given shape and an ellipses. Some of the other results obtained relate strongly to the well-known measures and methods broadly used in shape based object analysis tasks. [ABSTRACT FROM AUTHOR]

• We propose an image decomposition method based on pixel-wise analysis sparsity model, which utilizes task-aware priors to estimate a penalty weight for each pixel of the transformed image generated with analysis operator. • We introduce new sparse penalty weight estimation strategy for the two vision tasks: rain streaks removal and Retinex decomposition. • The synthesis sparsity model is combined with analysis sparsity model to code one separation layer. The mechanism could model the layer that can be further refined into new layers, and thus could address more than one issues simultaneously. Decomposing an image into two 'simpler' layers has been widely used in low-level vision tasks, such as image recovery and enhancement. It is an ill-posed problem since the number of unknowns are larger than the input. In this paper, a two-step strategy is introduced, including task-aware priors estimate and a decomposition model. A pixel-wise analysis sparsity model is proposed to regularize the separation layers, which supposes the transformed image generated with analysis operator is sparse. Unlike regularizing all pixels with one penalty weight, we try to estimate each pixel's sparsity level with task-aware priors and to achieve pixel-wise sparse penalty. Additionally, one separation layer is regularized with both synthesis sparsity model and pixel-wise analysis sparsity model to exploit their complementary mechanisms. Unlike the analysis one utilizing image local features, the synthesis one exploits an over-complete dictionary and non-local similarity cues to provide flexible prior for regularizing the decomposition results. The proposed model is solved by an alternating optimization algorithm. We evaluate it with two applications, Retinex model and rain streaks removal. Extensive experiments on multiple enhancement datasets, many synthetic and real rainy images demonstrate that our method can remove imaging noise during Retinex decomposition, and can produce high fidelity deraining results. It achieves competing performance in terms of quantitative metrics and visual quality compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• We propose a simultaneous robust matching pursuit (SRMP) method. • We devise an efficient half-quadratic optimization algorithm for SRMP. • We put forward a SRMP based multi-view subspace clustering approach. • A SRMP based classifier is developed for multi-view pattern classification. Joint sparse representation (JSR) has attracted massive attention with many successful applications in pattern recognition recently. In this paper, we propose a novel robust multi-view JSR method referred to as Simultaneous Robust Matching Pursuit (SRMP) based on the outlier-resistant M-estimator originating from robust statistics. Because of the complexity of the objective function, we design an efficient optimization algorithm to implement SRMP based on the half-quadratic theory. In addition, we have also extended the proposed method for the problems of multi-view subspace clustering and multi-view pattern classification, respectively. The experimental results corroborate the efficacy and robustness of SRMP for multi-view data recovery, subspace clustering and classification. [ABSTRACT FROM AUTHOR]

Multi-view data with each view corresponding to a type of feature set are common in real world. Usually, previous multi-view learning methods assume complete views. However, multi-view data are often incomplete, namely some samples have incomplete feature sets. Besides, most data are unlabeled due to a large cost of manual annotation, which makes learning of such data a challenging problem. In this paper, we propose a novel subspace learning framework for incomplete and unlabeled multi-view data. The model directly optimizes the class indicator matrix, which establishes a bridge for incomplete feature sets. Besides, feature selection is considered to deal with high dimensional and noisy features. Furthermore, the inter-view and intra-view data similarities are preserved to enhance the model. To these ends, an objective is developed along with an efficient optimization strategy. Finally, extensive experiments are conducted for multi-view clustering and cross-modal retrieval, achieving the state-of-the-art performance under various settings. [ABSTRACT FROM AUTHOR]

Over the past decade, salient object detection has attracted a lot of interests in computer vision. Although many models have been proposed to detect the salient object in an arbitrary image, this problem is still plagued with complex backgrounds and scattered objects. To address this issue, in this paper, we explore the information in cross features via a diversity-induced multi-view regularization under the Hilbert-Schmidt Independence Criterion (HSIC). Based on the diversity term, a new matrix decomposition based model is proposed for salient object detection. Furthermore, S 1 / 2 regularizer is introduced to constrain the background part. This regularizer will make the background much cleaner in the saliency map. A group sparsity induced norm is imposed on the salient part in order to involve the potential spatial relationships of image patches. Our method is solved through an augmented Lagrange multipliers method, and high-level priors are also integrated to boost the performance. Experiments on the four widely used datasets show that our method outperforms the state-of-the-art models. [ABSTRACT FROM AUTHOR]

The multi-instance dictionary plays a critical role in multi-instance data representation. Meanwhile, different multi-instance learning applications are evaluated by specific multivariate performance measures. For example, multi-instance ranking reports the precision and recall. It is not difficult to see that to obtain different optimal performance measures, different dictionaries are needed. This observation motives us to learn performance-optimal dictionaries for this problem. In this paper, we propose a novel joint framework for learning the multi-instance dictionary and the classifier to optimize a given multivariate performance measure, such as the F 1 score and precision at rank k . We propose to represent the bags as bag-level features via the bag-instance similarity, and learn a classifier in the bag-level feature space to optimize the given performance measure. We propose to minimize the upper bound of a multivariate loss corresponding to the performance measure, the complexity of the classifier, and the complexity of the dictionary, simultaneously, with regard to both the dictionary and the classifier parameters. In this way, the dictionary learning is regularized by the performance optimization, and a performance-optimal dictionary is obtained. We develop an iterative algorithm to solve this minimization problem efficiently using a cutting-plane algorithm and a coordinate descent method. Experiments on multi-instance benchmark data sets show its advantage over both traditional multi-instance learning and performance optimization methods. [ABSTRACT FROM AUTHOR]

In recent years, many representation based classifications have been proposed and widely used in face recognition. However, these methods code and classify testing images separately even for image-set of the same subject. This scheme utilizes only an individual representation rather than the collective one to classify such a set of images, doing so obviously ignores the correlation among the given set of images. In this paper, a joint representation classification (JRC) for collective face recognition is presented. JRC takes the correlation of multiple images as well as a single representation into account. Even for an image-set mixed with different subjects, JRC codes all the testing images over the base images simultaneously to facilitate recognition. To this end, the testing images are aligned into a matrix and the joint representation coding is formulated as a generalized l 2 , q − l 2 , p matrix minimization problem. A unified algorithm, named by iterative quadratic method (IQM), and its practical implementation are developed specially to solve the induced optimization problem for any q ∈ [ 1 , 2 ] and p ∈ ( 0 , 2 ] . Experimental results on three public databases show that the JRC with practical IQM not only saves much computational cost but also achieves better performance in collective face recognition than state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Biclustering represents an intrinsically complex problem, where the aim is to perform a simultaneous row- and column-clustering of a given data matrix. Some recent approaches model this problem using factor graphs, so to exploit their ability to open the door to efficient optimization approaches for well designed function decompositions. However, while such models provide promising results, they do not scale to data matrices of reasonable size. In this paper, we take a step towards addressing this issue, by proposing a novel approach to biclustering based on factor graphs, which yields high quality solutions and scales more favorably than previous methods. Specifically, we cast biclustering as the sequential search for a single bicluster, and propose a binary and compact factor graph that can be solved efficiently using the max-sum algorithm. The proposed approach has been tested and compared with state-of-the-art methods on four datasets (two synthetic and two real world data), providing encouraging results with respect both to previous approaches based on factor graphs and to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

This paper addresses the weighted orthogonal Procrustes problem of matching stochastically perturbed point clouds, formulated as an optimization problem with a closed-form solution. A novel uncertainty characterization of the solution of this problem is proposed resorting to perturbation theory concepts, which admits arbitrary transformations between point clouds and individual covariance and cross-covariance matrices for the points of each cloud. The method is thoroughly validated through extensive Monte Carlo simulations, and particularly interesting cases where nonlinearities may arise are further analyzed. [ABSTRACT FROM AUTHOR]

Graph matching is a fundamental problem in computer vision area. Graph matching problem that incorporates pair-wise constraints can be formulated as an integer quadratic programming (IQP) problem with affine mapping constraint. Since it is known to be NP-hard, approximate relaxation methods are usually required to find approximate solutions. In this paper, we present a new effective graph matching relaxation method, called Lagrangian relaxation graph matching (LRGM), which aims to generate a relaxation model by incorporating the affine mapping constraint into the matching objective optimization. There are three main benefits of the proposed LRGM method: (1) The nonnegative affine mapping constraint encoding one-to-one mapping is naturally incorporated in LRGM relaxation via Lagrangian regularization. (2) By further adding a ℓ 1 -norm constraint, LRGM can generate a sparse solution empirically and thus returns a desired discrete solution for original IQP matching problem. (3) An effective update algorithm is derived to solve the proposed LRGM model. Theoretically, the converged solution can be proven to be Karush–Kuhn–Tucker (KKT) optimal. Experimental results on both synthetic data and real-world image datasets show the effectiveness and benefits of the proposed LRGM method. [ABSTRACT FROM AUTHOR]

Multiple Empirical Kernel Learning (MEKL) explicitly maps samples into different empirical feature spaces in which the kernel features of the mapped samples can be directly provided. Thus, MEKL is much easier than the conventional Multiple Kernel Learning (MKL) in terms of processing and analyzing the structure of mapped feature spaces. However, the computational complexity of MEKL with M empirical feature spaces is O ( MN 3 ) where N is the number of training samples. The dimensions of the generated empirical feature spaces are approximate to N . When dealing with large-scale problems, MEKL cannot handle them properly due to the severe computation and memory burden. Moreover, most existing MEKL utilizes the gradient decent optimization to learn classifiers, but it is time consuming for training. Therefore, this paper proposes a Multiple Random Empirical Kernel Learning Machine (MREKLM) to overcome these problems. The proposed MREKLM adopts the random projection idea to map samples into multiple low-dimensional empirical feature spaces with lower computational complexity O ( MP 3 ) , where P ( ≪ N ) is the number of the randomly selected samples. After that, MREKLM adopts an analytical optimization approach to directly deal with multi-class problems. The computational complexity of MREKLM is O ( M 3 P 3 ) . Experimental results also validate both efficiency and effectiveness of the proposed MREKLM. The contributions of this work are: (1) proposing a fast MEKL algorithm named MREKLM, (2) introducing an efficient random empirical kernel mapping approach, and (3) extending the capability of MEKL to handle large-scale problems. [ABSTRACT FROM AUTHOR]

This paper introduces the Graph Embedded One-Class Support Vector Machine and Graph Embedded Support Vector Data Description methods. These methods constitute novel extensions of the One-Class Support Vectors Machines and Support Vector Data Description, incorporating generic graph structures that express geometric data relationships of interest in their optimization process. Local or global relationships between the training patterns can be expressed with single graphs or combinations of fully connected and k NN graphs. We show that the adoption of generic geometric class information acts as a regularizer to the solution of the original methods. Moreover, we prove that the regularized solutions for both One-Class Support Vector Machine and Support Vector Data Description are equivalent to applying the original methods in a transformed (and shared) feature space. Qualitative and quantitative evaluation of the proposed methods shows that they compare favorably to the standard OC-SVM and SVDD classifiers, respectively. [ABSTRACT FROM AUTHOR]

In this paper, we present an approach that utilizes multiple exemplar agent-based motion models (AMMs) to extract motion features (representing crowd behaviors) from the captured crowd trajectories. In the exemplar-based framework, we propose an iterative optimization algorithm to measure the correlation between any exemplar AMM and the trajectory data. It is based on the Extended Kalman Smoother and KL-divergence. In addition, based on the proposed correlation measure, we introduce the novel individual feature, in combination with the holistic feature, to describe crowd motions. Our results show that the proposed features perform well in classifying real-world crowd scenes. [ABSTRACT FROM AUTHOR]

Extreme Learning Machine (ELM) is a promising model for training single-hidden layer feedforward networks (SLFNs) and has been widely used for classification. However, ELM faces the challenge of arbitrarily selected parameters, e.g. , the network weights and hidden biases. Therefore, many efforts have been made to enhance the performance of ELM, such as using evolutionary algorithms to explore promising areas of the solution space. Although evolutionary algorithms can explore promising areas of the solution space, they are not able to locate global optimum efficiently. In this paper, we present a new Memetic Algorithm (MA)-based Extreme Learning Machine (M-ELM for short). M-ELM embeds the local search strategy into the global optimization framework to obtain optimal network parameters. Experiments and comparisons on 46 UCI data sets validate the performance of M-ELM. The corresponding results demonstrate that M-ELM significantly outperforms state-of-the-art ELM algorithms. [ABSTRACT FROM AUTHOR]

• We address a novel task named "group activity trajectory recovery" and screen suspected high-risk patients in the population by combining GPS data and video data to track COVID-19 contacts in an epidemiological survey. It combines efficiency and accuracy compared to existing methods, and the actual operation is more reliable and efficient. • To address the problem of the inaccuracy of existing GPS positioning for tracking confirmed patients, this paper combines GPS data and video data to achieve tracking of close contact patients and proposes a combined optimization method (CO-SC). • A high-quality GATR-GPS dataset is constructed to simulate the tracking of close contact patients in realistic scenarios, and it is verified that the proposed method can significantly improve the performance of close contact patient tracking. • Randomly take the GPS data and video data at the same moment, calculate their errors, traverse them 10 times, and average them to get the average error of the two kinds of data. The average error of GPS data is over 200m, while the error of video data is within 1m. • In the self-built dataset and all using GPS data, our method obtains 89.80% for Rank 1 and 80.15% for mAP, which is a 7.35% improvement on Rank 1 and a 5.81% improvement on mAP compared to the CADL method. Contact tracking plays an important role in the epidemiological investigation of COVID-19, which can effectively reduce the spread of the epidemic. As an excellent alternative method for contact tracking, mobile phone location-based methods are widely used for locating and tracking contacts. However, current inaccurate positioning algorithms that are widely used in contact tracking lead to the inaccurate follow-up of contacts. Aiming to achieve accurate contact tracking for the COVID-19 contact group, we extend the analysis of the GPS data to combine GPS data with video surveillance data and address a novel task named group activity trajectory recovery. Meanwhile, a new dataset called GATR-GPS is constructed to simulate a realistic scenario of COVID-19 contact tracking, and a coordinated optimization algorithm with a spatio-temporal constraint table is further proposed to realize efficient trajectory recovery of pedestrian trajectories. Extensive experiments on the novel collected dataset and commonly used two existing person re-identification datasets are performed, and the results evidently demonstrate that our method achieves competitive results compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• Current momentum-based methods usually suffer from the transferability saturation dilemma, which may degrade their capacity to break black-box models. • To mitigate the transferability saturation effect, we propose cyclical optimization that divides the generation process into multiple phases and treats the velocity from the previous phase as helpful knowledge to guide a new attack? We design cyclical augmentation algorithm for further improving fooling rates by stabilizing update directions and learning diverse boundary information without additional retraining costs. • Cyclical optimization and cyclical augmentation enhance black-box adversarial examples from the perspectives of the optimization and augmentation, respectively. We can apply them simultaneously to achieve more superior performance, referred to as cyclical adversarial attack. • Our method has well generalizability to integrate with existing methods, and establishes state-of-the-art for transferable attacks against normally trained models and defenses under both standalone and combination settings Deep neural networks (DNNs) have shown vulnerability to adversarial attacks. By exploiting the transferability of adversarial examples, attackers can fool models under black-box settings without accessing the underlying information. However, they often exhibit weak performance when transferring to defenses, which may give a false sense of security. In this paper, we propose Cyclical Adversarial Attack (CA 2), a general and straightforward method to boost the transferability to break defenders. We first revisit the momentum-based methods from the perspective of optimization and find that they usually suffer from the transferability saturation dilemma. To address this, CA 2 performs cyclical optimization algorithm to produce adversarial examples. Unlike the standard momentum policy that accumulates the velocity to continuously update the solution, we divide the generation process into multiple phases and treat the velocity vectors from the previous phase as proper knowledge to guide a new adversarial attack with larger steps. Moreover, CA 2 applies a novel and compatible augmentation algorithm at every optimization in a loop manner for enhancing the black-box transferability further, referred to as cyclical augmentation. Extensive experiments conducted on a variety of models not only validate the efficacy of each designed algorithm in CA 2 , but also illustrate the superiority of our method compared with the state-of-the-art transferable attacks. Our implemental code is publicly available at https://github.com/mesunhlf/CA2. [ABSTRACT FROM AUTHOR]

• By regarding multi-view learning as a natural choice to address feature heterogeneity in federated setting, we present a novel federated multi-view learning method based on orthogonal matrix factorization due to its extensibility and lower costs. • By allowing each node with the flexibility to address its subproblem approximately, the proposed optimization algorithm is able to handle communication variability associated with node heterogeneity. • We further provide convergence guarantees for the proposed algorith that also take the communication variability into consideration and provide insight into the implementation performance. • Empirical study shows encouraging results of our model in comparison to several state-of-the-art algorithms, as we illustrate through simulations on benchmark federated multi-view datasets. Multi-view learning aims to explore a global common structure shared by different views collected from multiple individual sources. The nascent field of federated learning tries to learn a global model over distributed networks of devices. This paper shows that multi-view learning is naturally suited to address the feature heterogeneity of the federated setting. We propose a novel model, namely robust federated multi-view learning (FedMVL), which is considered in the following formulation: given a dataset with M views, it is required to train machine learning models while the M views are distributed across M devices or nodes. Considering the unique challenges like stragglers and fault tolerance in federated setting, we derive an iterative federated optimization algorithm that allows each node with the flexibility to approximately address its subproblem. To the best of our knowledge, our model for the first time considers the issues including high communication cost, fault tolerance, and stragglers for distributed multi-view learning. The proposed model also achieves encouraging performance on clustering task compared to closely related methods, as we illustrate through simulations on several real-world datasets. [ABSTRACT FROM AUTHOR]

Abstract: This paper concerns the use of prototype reduction schemes (PRS) to optimize the computations involved in typical k-nearest neighbor (k-NN) rules. These rules have been successfully used for decades in statistical pattern recognition (PR) applications and are particularly effective for density estimation, classification, and regression because of the known error bounds that they possess. For a given data point of unknown identity, the k-NN possesses the phenomenon that it combines the information about the samples from a priori target classes (values) of selected neighbors to predict the target class of the tested sample, or to estimate the density function value of the given queried sample. Recently, an implementation of the k-NN, named as the locally linear reconstruction (LLR) , has been proposed. The salient and brilliant feature of the latter is that by invoking a quadratic optimization process, it is capable of systematically setting model parameters, such as the number of neighbors (specified by the parameter, k) and the weights. However, the LLR takes more time than other conventional methods when it has to be applied to classification tasks. To overcome this problem, we propose a strategy of using a PRS to efficiently compute the optimization problem. In this paper, we demonstrate, first of all, that by completely discarding the points not included by the PRS, we can obtain a reduced set of sample points, using which, in turn, the quadratic optimization problem can be computed far more expediently. The values of the corresponding indices are comparable to those obtained with the original training set (i.e., the one which considers all the data points) even though the computations required to obtain the prototypes and the corresponding classification accuracies are noticeably less. The proposed method has been tested on artificial and real-life data sets, and the results obtained are very promising, and could have potential in PR applications. [Copyright &y& Elsevier]

Abstract: Class imbalance limits the performance of most learning algorithms since they cannot cope with large differences between the number of samples in each class, resulting in a low predictive accuracy over the minority class. In this respect, several papers proposed algorithms aiming at achieving more balanced performance. However, balancing the recognition accuracies for each class very often harms the global accuracy. Indeed, in these cases the accuracy over the minority class increases while the accuracy over the majority one decreases. This paper proposes an approach to overcome this limitation: for each classification act, it chooses between the output of a classifier trained on the original skewed distribution and the output of a classifier trained according to a learning method addressing the course of imbalanced data. This choice is driven by a parameter whose value maximizes, on a validation set, two objective functions, i.e. the global accuracy and the accuracies for each class. A series of experiments on ten public datasets with different proportions between the majority and minority classes show that the proposed approach provides more balanced recognition accuracies than classifiers trained according to traditional learning methods for imbalanced data as well as larger global accuracy than classifiers trained on the original skewed distribution. [Copyright &y& Elsevier]

Abstract: In this paper, a so-called minimum class locality preserving variance support machine (MCLPV_SVM) algorithm is presented by introducing the basic idea of the locality preserving projections (LPP), which can be seen as a modified class of support machine (SVM) and/or minimum class variance support machine (MCVSVM). MCLPV_SVM, in contrast to SVM and MCVSVM, takes the intrinsic manifold structure of the data space into full consideration and inherits the characteristics of SVM and MCVSVM. We discuss in the paper the linear case, the small sample size case and the nonlinear case of the MCLPV_SVM. Similar to MCVSVM, the MCLPV_SVM optimization problem in the small sample size case is solved by using dimensionality reduction through principal component analysis (PCA) and one in the nonlinear case is transformed into an equivalent linear MCLPV_SVM problem under kernel PCA (KPCA). Experimental results on real datasets indicate the effectiveness of the MCLPV_SVM by comparing it with SVM and MCVSVM. [Copyright &y& Elsevier]

Abstract: Distance metric is a key issue in many machine learning algorithms. This paper considers a general problem of learning from pairwise constraints in the form of must-links and cannot-links. As one kind of side information, a must-link indicates the pair of the two data points must be in a same class, while a cannot-link indicates that the two data points must be in two different classes. Given must-link and cannot-link information, our goal is to learn a Mahalanobis distance metric. Under this metric, we hope the distances of point pairs in must-links are as small as possible and those of point pairs in cannot-links are as large as possible. This task is formulated as a constrained optimization problem, in which the global optimum can be obtained effectively and efficiently. Finally, some applications in data clustering, interactive natural image segmentation and face pose estimation are given in this paper. Experimental results illustrate the effectiveness of our algorithm. [Copyright &y& Elsevier]