Search

Multimodal classification of social media is used to classify data from different modalities into different categories, which is essential for understanding user behavior on the web. In this paper, we focus on classifying image-text pairs, specifically user-generated content on social media. Recently, the transformer network, a kind of self-attention network, has been widely studied in the disciplines of visual computing and language processing. In the attention mechanism, positive correlation is considered. However, multimedia content posted on social media is diverse. Images and text are not always consistent, and contrary information is also helpful for representation. Therefore, it is equally important to detect conflicts based on negative or inverse attention. Inspired by the attention mechanism, we propose a novel model, namely Crossmodal Bipolar Attention Network (CBAN). Different from existing positive dot-product and additive attention mechanisms, we propose a bipolar attention mechanism, which fuses visual and textual information through their direct and inverse semantic relationships to classify multimodal data. We conducted experiments on multiple multimodal classification data sets, for performing sentiment analysis, sarcasm detection, crisis categorization and hate-speech detection. Experimental results show that our proposed CBAN consistently outperforms state-of-the-art methods in all classification tasks. [ABSTRACT FROM AUTHOR]

Pavement crack detection is an essential defect inspection task to ensure traffic and driving safety. Despite the recent studies that have achieved promising performance for inspecting pavement cracks, they still suffer from the challenges of background complexity, crack diversity, and generalization ability. To address these problems, we propose a contrastive decoupling network (CDNet) for detecting pavement cracks. Specifically, this contrastive decoupling framework separately extracts the global and local features with contrastive learning. It can effectively boost the discriminative power and generalization ability of the feature representations. Moreover, we propose a global semantic enhancement (GSE) module to enhance the semantic information of the global features. This module can reinforce the distinguishability for accurately identifying cracks and backgrounds. Furthermore, we propose a local detail refinement (LDR) module to refine the detailed information of the local features. This module can strengthen the localizability for detecting cracks with precise shapes. In addition, we propose a dynamic dependency-aware feature aggregation (DDFA) scheme to adaptively integrate the global and local features based on contextual dependencies. This scheme can enrich the output features for effectively detecting cracks in each image. Experimental results on four pavement crack detection datasets, namely Crack500, CrackTree200, CFD, and AEL, demonstrate that our proposed method outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The emergence of wearable devices has opened up new potentials for egocentric activity recognition. Although some methods integrate attention mechanisms into deep neural networks to capture fine-grained human-object interactions in a weak-supervision manner, they either ignore exploiting the temporal consistency or generate attention based on considering appearance cues only. To address these limitations, in this paper, we propose an enhanced attention-tracking method, combined with multi-branch network (EAT-MBNet), for egocentric activity recognition. Specifically, we propose class-aware attention maps (CAAMs) by employing a self-attention-based module to refine the class activation maps (CAMs). Our proposed method can enhance the semantic dependency between the activity categories and the feature maps. To highlight the discriminative features from the regions of interest across frames, we propose a flow-guided attention-tracking (F-AT) module, by simultaneously leveraging historical attention and motion patterns. Furthermore, we propose a cross-modality modeling branch based on an interactive GRU module, which captures the time-synchronized long-term relationships between the appearance and motion branches. Experimental results on four egocentric activity benchmarks demonstrate that the proposed method achieves state-of-the-art performance. [ABSTRACT FROM AUTHOR]

The growth in social media has led to the increasing spread of unverified or false information. Automatically detecting rumours and accessing their veracity, i.e., false rumours, true rumours, or unverified rumours, is an important and challenging task in social media analytics. This paper aims to build an effective and scalable stream classification framework for early fine-grained rumour classification based on community response. We propose a Causal Diffused Graph-Transformer Network (CDGTN) to extract features from the source-reply graph in a social media conversation. Then, we propose Source-Guided Incremental Attention Pooling (SGIAP) to aggregate the encoded features with discrete timestamps. To improve the performance of early classification, we propose a Stacked Early Classification Loss (SecLoss), which aims to minimize the classification loss over the time instances. This can greatly improve the effectiveness of early classification of rumours. To improve the efficiency of streaming rumour verification, we propose a continued inference algorithm based on prefix-sum, which can greatly reduce the computational complexity of stream classification of rumours. Furthermore, we annotated the first Chinese rumour verification dataset, by extending the existing Chinese-Twitter dataset, namely CR-Twitter, originally for rumour detection. We conducted experiments on the Twitter15, Twitter16, PHEME, Weibo, and the extended CR-Twitter datasets for rumour classification, to verify our proposed stream classification framework. The experimental results show that our proposed framework can significantly boost the effectiveness and efficiency of early stream classification of rumours. Models and datasets are released at: https://thcheung.github.io/cdgtn/. [ABSTRACT FROM AUTHOR]

Abstract Recent random-forest (RF)-based image super-resolution approaches inherit some properties from dictionary-learning-based algorithms, but the effectiveness of the features working in RF is overlooked in the literature. In this paper, we present a novel feature-augmented random forest (FARF) method for image super-resolution, where the conventional gradient-based features are proposed to augment the features used in RF, and different feature recipes are formulated on different processing stages in an RF. The advantages of our method are that, firstly, the dictionary-learning-based features are enhanced by adding gradient magnitudes, based on the observation that the non-linear gradient magnitudes are highly discriminative. Secondly, generalized locality-sensitive hashing (LSH) is used to replace principal component analysis (PCA) for feature dimensionality reduction in constructing the trees, but the original high-dimensional features are employed, instead of the compressed LSH features, for the leaf-nodes' regressors. With the use of the original higher dimensional features, the regressors can achieve better learning performances. Finally, we present a generalized weighted ridge regression (GWRR) model for the leaf-nodes' regressors. Experiment results on several public benchmark datasets show that our FARF method can achieve an average gain of about 0.3 dB, compared to traditional RF-based methods. Furthermore, a fine-tuned FARF model can compare to, or (in many cases) outperform, some recent state-of-the-art deep-learning-based algorithms. Highlights • Recent random-forest (RF)-based image super-resolution approaches inherit some properties from dictionary-learning-based algorithms, but the effectiveness of the features working in RF is overlooked in the literature. • In this paper, we present a novel feature-augmented random forest (FARF) method for image super-resolution, where the non-linear gradient magnitudes are proposed to augment the features used in RF, generalized locality-sensitive hashing (LSH) is used to replace principal component analysis (PCA) for feature dimensionality, and different feature recipes are formulated on different processing stages in an RF. [ABSTRACT FROM AUTHOR]

Highlights • A hierarchical framework is proposed to mimic humans for handling image tags. • A tree structure is constructed for image labels using a learning method. • Our model treats simple labels and complex labels separately. • Experiments results show promising performance on different datasets. Abstract Artificial Intelligence has attracted much of researchers' attention in recent years. A question we always ask is: "Can machines replace human beings to some extent?" This paper aims to explore the knowledge learning for an image-annotation framework, which is an easy task for humans but a tough task for machines. This paper's research is based on an assumption that machines have two systems of thinking, each of which handles the labels of images at different abstract levels. Based on this, a new hierarchical model for image annotation is introduced. We explore not only the relationships between the labels and the features used, but also the relationships between labels. More specifically, we divide labels into several hierarchies for efficient and accurate labeling, which are constructed using our Associative Memory Sharing method, proposed in this paper. [ABSTRACT FROM AUTHOR]

Highlights • This paper provides a systematic review of current histogram-based local feature descriptors, which have been applied for facial-expression recognition. • The weaknesses and strengths of the existing descriptors are discussed and analysed. • A comprehensive evaluation of the performances of 27 local descriptors is presented. • The best overall performances are obtained by LPQ and LGBPHS. This paper aims to present histogram-based local descriptors applied to Facial Expression Recognition (FER) from static images and provide a systematic review and analysis of them. First, we describe the main steps in encoding binary patterns in a local patch, which are required in every histogram-based local descriptor. Then, we list the existing local descriptors, while analysing their strengths and weaknesses. Finally, we present the experimental results of all these descriptors on commonly used facial expression databases, with varying resolution, noise, occlusion, and number of sub-regions, as well as comparing them with the results obtained by the state-of-the-art deep learning methods. This paper aims to bring together different studies of the visual features for FER by evaluating their performances under the same experimental setup, and critically reviewing various classifiers making use of the local descriptors. [ABSTRACT FROM AUTHOR]

Compliance control strategies have been utilised for the ultraprecision polishing process for many years. Most researchers execute active compliance control strategies by employing impedance control law on a robot development platform. However, these methods are limited by the load capacity, positioning accuracy, and repeatability of polishing mechanisms. Moreover, a sophisticated actuator mounted at the end of the end-effector of robots is difficult to maintain in the polishing scenario. In contrast, a hybrid mechanism for polishing that possesses the advantages of serial and parallel mechanisms can mitigate the above problems, especially when an active compliance control strategy is employed. In this research, a high-frequency-impedance robust force control strategy is proposed. It outputs a position adjustment value directly according to a contact pressure adjustment value. An open architecture control system with customised software is developed to respond to external interrupts during the polishing procedure, implementing the active compliance control strategy on a hybrid mechanism. Through this method, the hybrid mechanism can adapt to the external environment with a given contact pressure automatically instead of relying on estimating the environment stiffness. Experimental results show that the proposed strategy adapts the unknown freeform surface without overshooting and improves the surface quality. The average surface roughness value decreases from 0.057 um to 0.027 um. [ABSTRACT FROM AUTHOR]

We propose a random-forest scheme, namely joint maximum purity forest (JMPF), for classification, clustering, and regression tasks. In the JMPF scheme, the original feature space is transformed into a compactly preclustered feature space, via a trained rotation matrix. The rotation matrix is obtained through an iterative quantization process, where the input data, inclined to different classes, is clustered to the respective vertices of the feature space with maximum purity. In the feature space, orthogonal hyperplanes, which are employed at the split nodes of the decision trees in a random forest, can effectively tackle the clustering problems. We evaluated our proposed method on public benchmark datasets for regression and classification tasks, and experiments showed that JMPF remarkably outperforms other state-of-the-art random-forest-based approaches. Furthermore, we applied JMPF to image super-resolution (SR) specifically because the transformed, compact features are more discriminative to the clustering-regression scheme. Experimental results on several public datasets also showed that the JMPF-based image SR scheme is consistently superior to recent state-of-the-art image SR algorithms. [ABSTRACT FROM AUTHOR]

Face recognition across age progression remains one of the area's most challenging tasks, as the aging process affects both the shape and texture of a face. One possible solution is to apply a probabilistic model to represent a face simultaneously with its identity variable, which is stable through time, and its aging variable, which changes with time. However, as the aging process varies for different people, a person may look younger or older than another person, even though their ages are the same. Consequently, using the ‘real’ age labels given by existing face datasets for age-invariant face recognition will inevitably introduce ambiguity to learning algorithms. In this paper, an identity-inference model, based on age-subspace learning from appearance-age labels, is proposed. We first model human identity and aging variables simultaneously using Probabilistic Linear Discriminant Analysis (PLDA). Then, the aging subspace is learnt independently with the appearance-age labels, and the identity subspace is then determined iteratively with the Expectation-Maximization (EM) algorithm. We found that the learned aging subspace is insensitive to the training face images used, and is independent of the identity model. Consequently, the recognition of aging faces becomes simpler as identity inference no longer needs to consider age labels. Furthermore, in our algorithm, different identity features learnt from the identity model are further combined using Canonical Correlation Analysis (CCA), where their correlations are maximized for face recognition. A thorough experimental analysis of face recognition is performed on three public domain face-aging datasets: FGNET, MORPH, and CACD. Experiment results show that the proposed framework can achieve a comparable, or even better, performance against other state-of-the-art methods, especially when the age range is large. [ABSTRACT FROM AUTHOR]

Aim: The aim of the present study was to determine the practice patterns in the management of anterior urethral strictures among urologists in Hong Kong. Patients and Methods: A 14‐item questionnaire was delivered either directly or by mail to all 126 registered urologists in Hong Kong. Information including demographic data, number of anterior urethral strictures treated, diagnostic methods, treatment options and follow‐up strategy were collected. The questionnaire also asked for the personal opinion about the treatment plan of two stricture case scenarios. Results: The response rate was 48 per cent. The majority (87 per cent) of urologists treated <10 anterior urethral strictures per year. Minimal invasive procedures, including urethral dilatation using metal sounds (77 per cent) or cystoscopy‐guided (74 per cent) and direct visual internal urethrotomy (57 per cent), were more commonly performed by urologists to treat urethral strictures. The majority of urologists (82.6 per cent) performed less than five urethroplasties per year. In the two case scenarios of long bulbous urethral stricture and recurrent short urethral stricture, approximately 10 per cent of respondents would refer the cases to other urologists and approximately 75 per cent would choose to perform reconstructive surgeries. The remaining 15 per cent of respondents would choose minimally‐invasive procedures for these strictures. Nearly two‐thirds (62.3 per cent) of urologists believed that urethroplasty should be proposed only after failed endoscopic treatment. Workups for urethral stricture disease were consistent, while the modalities to access the outcome were highly heterogeneous. Conclusion: In Hong Kong, the majority of urologists choose to perform urethroplasty for long bulbous urethral strictures and recurrent short bulbous urethral strictures in a case scenario situation. However, in actual practice, most perform less than five urethroplasties per year. A small caseload, lack of experience and understanding of urethral reconstructive surgery means that most urologists in Hong Kong would hesitate to carry out primary urethroplasty in correctly‐selected patients for whom primary reconstruction would have been the treatment of choice. [ABSTRACT FROM AUTHOR]

Among the challenges faced by current active shape or appearance models, facial-feature localization in the wild, with occlusion in a novel face image, i.e. in a generic environment, is regarded as one of the most difficult computer-vision tasks. In this paper, we propose an Active Appearance Model (AAM) to tackle the problem of generic environment. Firstly, a fast face-model initialization scheme is proposed, based on the idea that the local appearance of feature points can be accurately approximated with locality constraints. Nearest neighbors, which have similar poses and textures to a test face, are retrieved from a training set for constructing the initial face model. To further improve the fitting of the initial model to the test face, an orthogonal CCA (oCCA) is employed to increase the correlation between shape features and appearance features represented by Principal Component Analysis (PCA). With these two contributions, we propose a novel AAM, namely the shape-appearance-correlated AAM (SAC-AAM), and the optimization is solved by using the recently proposed fast simultaneous inverse compositional (Fast-SIC) algorithm. Experiment results demonstrate a 5–10% improvement on controlled and semi-controlled datasets, and with around 10% improvement on wild face datasets in terms of fitting accuracy compared to other state-of-the-art AAM models. [ABSTRACT FROM AUTHOR]

A two-step face-hallucination framework is proposed to reconstruct a high-resolution (HR) version of a face from an input low-resolution (LR) face, based on learning from LR-HR example face pairs using orthogonal canonical correlation analysis (orthogonal CCA) and linear mapping. In the proposed algorithm, face images are first represented using principal component analysis (PCA). Canonical correlation analysis (CCA) with the orthogonality property is then employed, to maximize the correlation between the PCA coefficients of the LR and the HR face pairs to improve the hallucination performance. The original CCA does not own the orthogonality property, which is crucial for information reconstruction. We propose using orthogonal CCA, which is proven by experiments to achieve a better performance in terms of global face reconstruction. In addition, in the residual-compensation process, a linear-mapping method is proposed to include both the interand intrainformation about manifolds of different resolutions. Compared with other state-of-the-art approaches, the proposed framework can achieve a comparable, or even better, performance in terms of global face reconstruction and the visual quality of face hallucination. Experiments on images with various parameter settings and blurring distortions show that the proposed approach is robust and has great potential for real-world applications. ©2016SPIE and IS&T [DOI: 10.1117/1.JEI.25.3.033005] [ABSTRACT FROM AUTHOR]

In video surveillance, the captured face images are usually of low resolution (LR). Thus, a framework based on singular value decomposition (SVD) for performing both face hallucination and recognition simultaneously is proposed in this paper. Conventionally, LR face recognition is carried out by super-resolving the LR input face first, and then performing face recognition to identify the input face. By considering face hallucination and recognition simultaneously, the accuracy of both the hallucination and the recognition can be improved. In this paper, singular values are first proved to be effective for representing face images, and the singular values of a face image at different resolutions have approximately a linear relation. In our algorithm, each face image is represented using SVD. For each LR input face, the corresponding LR and high-resolution (HR) face-image pairs can then be selected from the face gallery. Based on these selected LR–HR pairs, the mapping functions for interpolating the two matrices in the SVD representation for the reconstruction of HR face images can be learned more accurately. Therefore, the final estimation of the high-frequency details of the HR face images will become more reliable and effective. The experimental results demonstrate that our proposed framework can achieve promising results for both face hallucination and recognition. [ABSTRACT FROM AUTHOR]

Similarity-measure-based spectrum matching is an effective approach to chemical compound identification. When the sizes of both the query library and the reference library become increasingly large, most existing spectrum-matching methods encounter a seriously heavy computation burden. In this paper, an effective and efficient compound-identification approach is proposed based on the frequency features of mass spectra. Considering the sparsity of mass spectra, a nonzero feature-selection strategy is proposed to decrease the feature dimensionality of mass spectra. To further improve its efficiency, a correlation-based filtering strategy is presented to select the most correlated reference spectra in order to create a reduced reference library. Based on the decreased features and the reduced reference library, the frequency-feature-based composite similarity measures are computed to estimate the chemical abstracts service (CAS) registry numbers of the mass spectra blue in a query library. Due to the reduction in both the feature dimensionality and the reference library, the computation time of the proposed method is only about 6%–11% of that of the existing methods, while the identification performance remains sufficiently competitive. Experimental results demonstrate the feasibility and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Establishing correct correspondences between two faces with different viewpoints has played an important role in 3D face reconstruction and other computer-vision applications. Usually, face images are considered to lack sufficient distinctive features to establish a large number of correspondences on uncalibrated images. In this paper, we investigate pore-scale facial features, which are formed from pores, fine wrinkles, and hair. These features have many characteristics that make them suitable for matching facial images under different variations. Using both biological observation and computer-vision consideration, a new framework is devised for pore-scale facial-feature extraction and matching. The matching difficulty under various skin appearances of different subjects and imaging distortion is also analyzed. For further improving the matching performance and tackling distortions such as varying illuminations and unfocused blurring, a pore-to-pore correspondences dataset is established for training a more distinctive and compact descriptor. Experiments are conducted on a face database containing 105 subjects, and the results prove that the pore-scale features are highly distinctive; face images with a minimum resolution of 600×700 (0.4 mega) pixels contain sufficient details to perform a reliable matching in different poses. Generally, our algorithm can establish between 500 and 2000 correct correspondences on a pair of uncalibrated face images of the same person. Furthermore, the proposed methods can be applied to face recognition, 3D reconstruction, etc. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, an efficient method based on singular values and potential-field representation is proposed for face-image retrieval. Firstly, we theoretically prove that the leading singular values of an image can be used as a rotation-shift-scale-invariant global feature. Then, for the feature-extraction stage, we exploit these special properties of the singular values to devise a compact, global feature for face-image representation. We also use the singular values of the potential field derived from edge gradients to enhance the retrieval performance. Experimental results based on the GTAV database show that the use of singular values as rotation-shift-scale-invariant global features is able to produce plausible retrieval results. [Copyright &y& Elsevier]

Abstract: For face recognition, image features are first extracted and then matched to those features in a gallery set. The amount of information and the effectiveness of the features used will determine the recognition performance. In this paper, we propose a novel face recognition approach using information about face images at higher and lower resolutions so as to enhance the information content of the features that are extracted and combined at different resolutions. As the features from different resolutions should closely correlate with each other, we employ the cascaded generalized canonical correlation analysis (GCCA) to fuse the information to form a single feature vector for face recognition. To improve the performance and efficiency, we also employ “Gabor-feature hallucination”, which predicts the high-resolution (HR) Gabor features from the Gabor features of a face image directly by local linear regression. We also extend the algorithm to low-resolution (LR) face recognition, in which the medium-resolution (MR) and HR Gabor features of a LR input image are estimated directly. The LR Gabor features and the predicted MR and HR Gabor features are then fused using GCCA for LR face recognition. Our algorithm can avoid having to perform the interpolation/super-resolution of face images and having to extract HR Gabor features. Experimental results show that the proposed methods have a superior recognition rate and are more efficient than traditional methods. [Copyright &y& Elsevier]

Abstract: In this paper, an efficient mapping model based on singular value decomposition (SVD) is proposed for face hallucination. We can observe and prove that the main singular values of an image at one resolution have approximately linear relationships with their counterparts at other resolutions. This makes the estimation of the singular values of the corresponding high-resolution (HR) face images from a low-resolution (LR) face image more reliable. From the signal-processing point of view, this can effectively preserve and reconstruct the dominant information in the HR face images. Interpolating the other two matrices obtained from the SVD of the LR image does not change either the primary facial structure or the pattern of the face image. The corresponding two matrices for the HR face images can be constructed in a “coarse-to-fine” manner using global reconstruction. Our proposed method retains the holistic structure of face images, while the learned mapping matrices, which are represented as embedding coefficients of the individual mapping matrices learned from LR-HR training pairs, can be seen as holistic constraints in the reconstruction of HR images. Compared to state-of-the-art algorithms, experiments show that our proposed face-hallucination scheme is effective in terms of producing plausible HR images with both a holistic structure and high-frequency details. [Copyright &y& Elsevier]

We propose to use the similarity between the sample instance and a number of exemplars as features in visual object detection. Concepts from multiple-kernel learning and multiple-instance learning are incorporated into our scheme at the feature level by properly calculating the similarity. The similarity between two instances can be measured by various metrics and by using the information from various sources, which mimics the use of multiple kernels for kernel machines. Pooling of the similarity values from multiple instances of an object part is introduced to cope with alignment inaccuracy between object instances. To deal with the high dimensionality of the multiple-kernel multiple-instance similarity feature, we propose a forward feature-selection technique and a coarse-to-fine learning scheme to find a set of good exemplars, hence we can produce an efficient classifier while maintaining a good performance. Both the feature and the learning technique have interesting properties. We demonstrate the performance of our method using both synthetic data and real-world visual object detection data sets. [ABSTRACT FROM PUBLISHER]

Marine hydrological elements are of vital importance in marine surveys. The evolution of these elements can have a profound effect on the relationship between human activities and marine hydrology. Therefore, the detection and explanation of the evolution laws of marine hydrological elements are urgently needed. In this paper, a novel method, named Evolution Trend Recognition (ETR), is proposed to recognize the trend of ocean fronts, being the most important information in the ocean dynamic process. Therefore, in this paper, we focus on the task of ocean-front trend classification. A novel classification algorithm is first proposed for recognizing the ocean-front trend, in terms of the ocean-front scale and strength. Then, the GoogLeNet Inception network is trained to classify the ocean-front trend, i.e., enhancing or attenuating. The ocean-front trend is classified using the deep neural network, as well as a physics-informed classification algorithm. The two classification results are combined to make the final decision on the trend classification. Furthermore, two novel databases were created for this research, and their generation method is described, to foster research in this direction. These two databases are called the Ocean-Front Tracking Dataset (OFTraD) and the Ocean-Front Trend Dataset (OFTreD). Moreover, experiment results show that our proposed method on OFTreD achieves a higher classification accuracy, which is 97.5%, than state-of-the-art networks. This demonstrates that the proposed ETR algorithm is highly promising for trend classification. [ABSTRACT FROM AUTHOR]

• A robust deep low-rank feature learning and encoding model for cross-age face recognition is proposed. • A manifold-constrained low-rank approximation algorithm is proposed to learn clean low-rank feature matrix for representation. • A locality-constrained feature encoding framework is proposed to represent the cross-age face images of the same identity by similar codewords. • The proposed method outperforms other state-of-the-art methods in terms of recognition rate. Cross-age face recognition (CAFR) is a challenging task, due to significant intra-personal variations. Furthermore, the training and testing data may contain random noise components. To address these issues, this paper proposes a deep low-rank feature learning and encoding method. Firstly, our method employs manifold learning in the low-rank optimization, which preserves the global and local structure of the data samples, while learning the clean low-rank features. Secondly, we encode the low-rank features using our locality-constrained feature encoding method, which learns an age-insensitive codebook from training data, and enables the intra-class samples to share the same local bases in a codebook. In the testing stage, the gallery and probe features are encoded by the learned codebook, which represents the images of the same identity by similar codewords for recognition. Furthermore, the periocular region of human faces is investigated for CAFR. Extensive experiments on five datasets demonstrate the effectiveness of our method. [ABSTRACT FROM AUTHOR]

Abstract: Image annotation tasks always lack accuracy and efficiency. Although many techniques that have been proposed in the last decade can give a reasonable performance, the large number of potential labels causes trouble in terms of decreasing the accuracy and efficiency. Both generative models and discriminative models have been proposed to solve the multi-label problem. Most of these complex models fail to achieve a good performance when they face an increasing number of image collections, with a dictionary that covers a large number of potential semantics. In this paper, we present a two-stage method for multi-class image labeling. We first introduce a simple label-filtering algorithm, which can remove most of the irrelevant labels for a query image while the potential labels are maintained. With a small population of potential labels left, we then explore the relationship between the features to be used and each single class. Hence, specific and effective features will be selected for each class to form a label-specific classifier. In other words, our approach prunes specific features for each single label and formalizes the annotation task as a discriminative classification problem. Experiments prove that our two-stage framework can achieve both efficiency and accuracy for image annotation. [Copyright &y& Elsevier]

The one-sample-per-person problem has become an active research topic for face recognition in recent years because of its challenges and significance for real-world applications. However, achieving relatively higher recognition accuracy is still a difficult problem due to, usually, too few training samples being available and variations of illumination and expression. To alleviate the negative effects caused by these unfavorable factors, in this paper we propose a more accurate spectral feature image-based 2DLDA (two-dimensional linear discriminant analysis) ensemble algorithm for face recognition, with one sample image per person. In our algorithm, multi-resolution spectral feature images are constructed to represent the face images; this can greatly enlarge the training set. The proposed method is inspired by our finding that, among these spectral feature images, features extracted from some orientations and scales using 2DLDA are not sensitive to variations of illumination and expression. In order to maintain the positive characteristics of these filters and to make correct category assignments, the strategy of classifier committee learning (CCL) is designed to combine the results obtained from different spectral feature images. Using the above strategies, the negative effects caused by those unfavorable factors can be alleviated efficiently in face recognition. Experimental results on the standard databases demonstrate the feasibility and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we propose an efficient algorithm to reconstruct the 3D structure of a human face from one or more of its 2D images with different poses. In our algorithm, the nonlinear least-squares model is first employed to estimate the depth values of facial feature points and the pose of the 2D face image concerned by means of the similarity transform. Furthermore, different optimization schemes are presented with regard to the accuracy levels and the training time required. Our algorithm also embeds the symmetrical property of the human face into the optimization procedure, in order to alleviate the sensitivities arising from changes in pose. In addition, the regularization term, based on linear correlation, is added in the objective function to improve the estimation accuracy of the 3D structure. Further, a model-integration method is proposed to improve the depth-estimation accuracy when multiple nonfrontal-view face images are available. Experimental results on the 2D and 3D databases demonstrate the feasibility and efficiency of the proposed methods. [ABSTRACT FROM PUBLISHER]

Abstract: In this paper, we propose a wavelet-based eigentransformation method for human face hallucination. Our algorithm uses the wavelet transform to decompose interpolated low-resolution (LR) images in the wavelet domain to obtain high-frequency information in three different directions, and employs the eigentransformation method to reconstruct the corresponding finer high-frequency content of the high-resolution (HR) images. The low-frequency content of the HR images in the wavelet domain is estimated based on the interpolated images directly. The resulting high-quality HR faces can be synthesized by using the inverse wavelet transform, with all the estimated coefficients. By combining interpolation and eigentransformation, the reconstructed images are less dependent on the training set selected, and can better preserve the low-frequency content. Thus, the reconstructed images look more like the ground-true HR images, as compared to the original eigentransformation method. Experimental results show that our proposed algorithm outperforms the original eigentransformation and other existing methods for face hallucination in terms of both visual quality and objective measurements. [Copyright &y& Elsevier]

Abstract: In this paper, we present a kernel-based eigentransformation framework to hallucinate the high-resolution (HR) facial image of a low-resolution (LR) input. The eigentransformation method is a linear subspace approach, which represents an image as a linear combination of training samples. Consequently, those novel facial appearances not included in the training samples cannot be super-resolved properly. To solve this problem, we devise a kernel-based extension of the eigentransformation method, which takes higher-order statistics of the image data into account. To generate HR face images with higher fidelity, the HR face image reconstructed using this kernel-based eigentransformation method is treated as an initial estimation of the target HR face. The corresponding high-frequency components of this estimation are extracted to form a prior in the maximum a posteriori (MAP) formulation of the SR problem so as to derive the final reconstruction result. We have evaluated our proposed method using different kernels and configurations, and have compared these performances with some current SR algorithms. Experimental results show that our kernel-based framework, along with a proper kernel, can produce good HR facial images in terms of both visual quality and reconstruction errors. [ABSTRACT FROM AUTHOR]

We have developed a new face hallucination framework termed from local pixel structure to global image super-resolution (LPS-GIS). Based on the assumption that two similar face images should have similar local pixel structures, the new framework first uses the input low-resolution (LR) face image to search a face database for similar example high-resolution (HR) faces in order to learn the local pixel structures for the target HR face. It then uses the input LR face and the learned pixel structures as priors to estimate the target HR face. We present a three-step implementation procedure for the framework. Step 1 searches the database for K example faces that are the most similar to the input, and then warps the K example images to the input using optical flow. Step 2 uses the warped HR version of the K example faces to learn the local pixel structures for the target HR face. An effective method for learning local pixel structures from an individual face, and an adaptive procedure for fusing the local pixel structures of different example faces to reduce the influence of warping errors, have been developed. Step 3 estimates the target HR face by solving a constrained optimization problem by means of an iterative procedure. Experimental results show that our new method can provide good performances for face hallucination, both in terms of reconstruction error and visual quality; and that it is competitive with existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we describe a video-object segmentation and 3D-trajectory estimation method for the analysis of dynamic scenes from a monocular uncalibrated view. Based on the color and motion information among video frames, our proposed method segments the scene, calibrates the camera, and calculates the 3D trajectories of moving objects. It can be employed for video-object segmentation, 2D-to-3D video conversion, video-object retrieval, etc. In our method, reliable 2D feature motions are established by comparing SIFT descriptors among successive frames, and image over-segmentation is achieved using a graph-based method. Then, the 2D motions and the segmentation result iteratively refine each other in a hierarchically structured framework to achieve video-object segmentation. Finally, the 3D trajectories of the segmented moving objects are estimated based on a local constant-velocity constraint, and are refined by a Hidden Markov Model (HMM)-based algorithm. Experiments show that the proposed framework can achieve a good performance in terms of both object segmentation and 3D-trajectory estimation. [ABSTRACT FROM AUTHOR]

Abstract: The H.264 video coding standard supports several inter-prediction coding modes using variable block sizes. The robust rate–distortion optimization (RDO) technique of both the motion estimation (ME) and the mode decision (MD) is adopted to achieve superior coding efficiency, which also entails a lot of complex computations. In this paper, by rearranging the order of candidate modes and using two new rate–distortion–complexity optimization (RDCO) functions, a more efficient RDCO framework for fast ME and MD is first proposed. Then, with this new framework, a novel complexity scalable control algorithm for H.264 inter-prediction coding is devised, which is based on an efficient complexity allocation and control (CAAC) scheme performed both at the frame level and the macroblock (MB) level. Experimental results show that our proposed algorithm can adaptively determine an appropriate cut-off point for a sequence of re-ordered candidate modes according to the current energy condition of a mobile device, which can adjust the complexity of inter-prediction coding to any appropriate level with minimum degradation in video quality. This can therefore prolong the operational lifetime of batteries. [Copyright &y& Elsevier]

Abstract: Example-based super-resolution is a promising approach to solving the image super-resolution problem. However, the learning process can be slow and prediction can be inaccurate. In this paper, we present a novel learning-based algorithm for image super-resolution to improve the computational speed and prediction accuracy. Our new method classifies image patches into several classes, for each class, a class-specific predictor is designed. A class-specific predictor takes a low-resolution image patch as input and predicts a corresponding high-resolution patch as output. The performances of the class-specific predictors are evaluated using different datasets formed by face images and natural-scene images. We present experimental results which demonstrate that the new method provides improved performances over existing methods. [Copyright &y& Elsevier]

Abstract: Face-detection methods based on cascade architecture have demonstrated a fast and robust performance. In most of these methods, each node of the cascade employs the simple Haar-like features from the central eye–nose–mouth region using the boosting method. However, it can be empirically observed that, in the deeper nodes of the boosting process, the non-face examples collected by bootstrapping are in fact very similar to the face examples, and the error rate of those feature-based weak classifiers is very close to 50%. Consequently, the performance of the face detector is hardly further improved. In this paper, we propose a novel and simple solution to this problem by imitating the characteristics of the human visual system. The main idea of our solution is to boost the cascade based on a hierarchical strategy, which employs the information from the central and surrounding parts of the face regions step by step. We argue that the context information about a face can be advantageously used in the deeper nodes of the boosting process when the features derived from the central region of the face do not provide any further benefit. Furthermore, we also propose a simplified Gabor feature to extend the feature set for the training of deeper nodes. Experiments show that our proposed method can improve not only the detection performance, but also the detection speed, by about 10% when compared to the original AdaBoost face-detection method for our implementation. [Copyright &y& Elsevier]

Abstract: In this paper, an efficient method for human facial expression recognition is presented. We first propose a representation model for facial expressions, namely the spatially maximum occurrence model (SMOM), which is based on the statistical characteristics of training facial images and has a powerful representation capability. Then the elastic shape–texture matching (ESTM) algorithm is used to measure the similarity between images based on the shape and texture information. By combining SMOM and ESTM, the algorithm, namely SMOM–ESTM, can achieve a higher recognition performance level. The recognition rates of the SMOM–ESTM algorithm based on the AR database and the Yale database are 94.5% and 94.7%, respectively. [Copyright &y& Elsevier]

This paper describes an efficient approach to scene-break detection, which can detect cuts, dissolves, and wipes reliably and effectively by means of temporally linear prediction models. In our algorithm, two linear prediction models are adopted to predict a current frame: one for dissolves, and the other for stationary scenes. The predicted frames, derived based on the two models, are compared with the original frames, and cuts and dissolves are then determined based on Bayesian cost functions. For the detection, our algorithm requires the setting of a single threshold only. In wipe detection, our linear prediction models are employed to detect areas of change between two successive frames. By accumulating the changed areas and the overlap of the changed areas over the successive frames, wipes of an arbitrary shape and direction are detected. Experimental results show that our algorithm can achieve a high level of precision even if a video contains object motion and camera motion. The detection time required to analyze a 38-mm video is no more than several seconds. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, an effective image magnification algorithm based on an adaptive Markov random field (MRF) model with a Bayesian framework is proposed. A low-resolution (LR) image is first magnified to form a high-resolution (HR) image using a fractal-based method, namely the multiple partitioned iterated function system (MPIFS). The quality of this magnified HR image is then improved by means of a blockwise adaptive MRF model using the Bayesian ‘maximum a posteriori’ (MAP) approach. We propose an efficient parameter estimation method for the MRF model such that the staircase artifact will be reduced in the HR image. Experimental results show that, when compared to the conventional MRF model, which uses a fixed set of parameters for a whole image, our algorithm can provide a magnified image with the well-preserved edges and texture, and can achieve a better PSNR and visual quality. [Copyright &y& Elsevier]

Abstract: In this paper, a new algorithm is proposed to derive the 3D structure of a human face from a group of face images under different poses. Based on the corresponding 2D feature points of the respective images, their respective poses and the depths of the feature points can be estimated based on measurements using the similarity transform. To accurately estimate the pose of and the 3D information about a human face, the genetic algorithm (GA) is applied. Our algorithm does not require any prior knowledge of camera calibration, and has no limitation on the possible poses or the scale of the face images. It also provides a means to evaluate the accuracy of the constructed 3D face model based on the similarity transform of the 2D feature point sets. Our approach can also be extended to face recognition to alleviate the effect of pose variations. Experimental results show that our proposed algorithm can construct a 3D face structure reliably and efficiently. [Copyright &y& Elsevier]

Abstract: In this paper, we propose a new face recognition algorithm based on a single frontal-view image for each face subject, which considers the effect of the face manifold structure. To compare two near-frontal face images, each face is considered a combination of a sequence of local image blocks. Each of the image blocks of one image can be reconstructed according to the corresponding local image block of the other face image. Then an elastic local reconstruction (ELR) method is proposed to measure the similarities between the image block pairs in order to measure the difference between the two face images. Our algorithm not only benefits from the face manifold structure, in terms of being robust to various image variations, but also is computationally simple because there is no need to build the face manifold. We evaluate the performance of our proposed face recognition algorithm with the use of different databases, which are produced under various conditions, e.g. lightings, expressions, perspectives, with/without glasses and occlusions. Consistent and promising experimental results were obtained, which show that our algorithm can greatly improve the recognition rates under all the different conditions. [Copyright &y& Elsevier]

Abstract: In this paper, a novel, elastic, shape-texture matching method, namely ESTM, for human face recognition is proposed. In our approach, both the shape and the texture information are used to compare two faces without establishing any precise pixel-wise correspondence. The edge map is used to represent the shape of an image, while the texture information is characterized by both the Gabor representations and the gradient direction of each pixel. Combining these features, a shape-texture Hausdorff distance is devised to compute the similarity of two face images. The elastic matching is robust to small, local distortions of the feature points such as those caused by facial expression variations. In addition, the use of the edge map, Gabor representations and the direction of the image gradient can all alleviate the effect of illumination to a certain extent. With different databases, experimental results show that our algorithm can always achieve a better performance than other face recognition algorithms under different conditions, except when an image is under poor and uneven illumination. Experiments based on the Yale database, AR database, ORL database and YaleB database show that our proposed method can achieve recognition rates of 88.7%, 97.7%, 78.3% and 89.5%, respectively. [Copyright &y& Elsevier]

Abstract: A novel algorithm based on spatial and statistical information is proposed for the display of high-dynamic range (HDR) images. In our proposed algorithm, an image is first decomposed into a base layer and a detailed layer, which represent its smoothed and fine details, respectively. The problem of overall impression preservation is regarded as a global issue in our algorithm. Statistical-based histogram adjustment is employed to deal with the base layer. The reproduction of visual details is regarded as a local issue. The detailed layer obtained using a spatial filter is adaptively enhanced according to the mapping function used for the base layer. The main contributions of our algorithm are that: (1) an adaptive detail-enhancement method is proposed and (2) a gain map is defined to combine the local and global issues. Experimental results show the superior performance of our approach in terms of visual quality. [Copyright &y& Elsevier]

Abstract: In this paper, an efficient representation method insensitive to varying illumination is proposed for human face recognition. Theoretical analysis based on the human face model and the illumination model shows that the effects of varying lighting on a human face image can be modeled by a sequence of multiplicative and additive noises. Instead of computing these noises, which is very difficult for real applications, we aim to reduce or even remove their effect. In our method, a local normalization technique is applied to an image, which can effectively and efficiently eliminate the effect of uneven illuminations while keeping the local statistical properties of the processed image the same as in the corresponding image under normal lighting condition. After processing, the image under varying illumination will have similar pixel values to the corresponding image that is under normal lighting condition. Then, the processed images are used for face recognition. The proposed algorithm has been evaluated based on the Yale database, the AR database, the PIE database, the YaleB database and the combined database by using different face recognition methods such as PCA, ICA and Gabor wavelets. Consistent and promising results were obtained, which show that our method can effectively eliminate the effect of uneven illumination and greatly improve the recognition results. [Copyright &y& Elsevier]

Abstract: The active shape model (ASM) has been used successfully to extract the facial features of a face image under frontal view. However, its performance degrades when the face concerned is under perspective variations. In this paper, a modified shape model is proposed which can adapt to face images under different orientations. To make the model represent a face more flexibly, the representations of the important facial features, i.e. the eyes, nose and mouth, and the face contour are separated. An energy function is defined that links up these two representations of a human face. In order to represent a face image under different poses, three models are employed to represent the important facial features: the left-viewed, right-viewed, and frontal-viewed models. The genetic algorithm (GA) is applied to search for the best representation of face images. Experimental results demonstrate that our proposed method can achieve a better performance in representing face images under different perspective variations and facial expressions than the conventional ASM can. [Copyright &y& Elsevier]

Abstract: The appearance of a face will vary drastically when the illumination changes. Variations in lighting conditions make face recognition an even more challenging and difficult task. In this paper, we propose a novel approach to handle the illumination problem. Our method can restore a face image captured under arbitrary lighting conditions to one with frontal illumination by using a ratio-image between the face image and a reference face image, both of which are blurred by a Gaussian filter. An iterative algorithm is then used to update the reference image, which is reconstructed from the restored image by means of principal component analysis (PCA), in order to obtain a visually better restored image. Image processing techniques are also used to improve the quality of the restored image. To evaluate the performance of our algorithm, restored images with frontal illumination are used for face recognition by means of PCA. Experimental results demonstrate that face recognition using our method can achieve a higher recognition rate based on the Yale B database and the Yale database. Our algorithm has several advantages over other previous algorithms: (1) it does not need to estimate the face surface normals and the light source directions, (2) it does not need many images captured under different lighting conditions for each person, nor a set of bootstrap images that includes many images with different illuminations, and (3) it does not need to detect accurate positions of some facial feature points or to warp the image for alignment, etc. [Copyright &y& Elsevier]

This paper proposes a novel illumination compensation algorithm, which can compensate for the uneven illuminations on human faces and reconstruct face images in normal lighting conditions. A simple yet effective local contrast enhancement method, namely block-based histogram equalization (BHE), is first proposed. The resulting image processed using BHE is then compared with the original face image processed using histogram equalization (HE) to estimate the category of its light source. In our scheme, we divide the light source for a human face into 65 categories. Based on the category identified, a corresponding lighting compensation model is used to reconstruct an image that will visually be under normal illumination. In order to eliminate the influence of uneven illumination while retaining the shape information about a human face, a 2D face shape model is used. Experimental results show that, with the use of principal component analysis for face recognition, the recognition rate can be improved by 53.3% to 62.6% when our proposed algorithm for lighting compensation is used. [Copyright &y& Elsevier]

The Gabor feature is effective for facial image representation, while linear discriminant analysis (LDA) can extract the most discriminant information from the Gabor feature for face recognition. In practice, the dimension of a Gabor feature vector is so high that the computation and memory requirements are prohibitively large. To reduce the dimension, one simple scheme is to extract the Gabor feature at sub-sampled positions, usually in a regular grid, in a face region. However, this scheme is not effective enough and degrades the recognition performance. In this paper, we propose a method to determine the optimal position for extracting the Gabor feature such that the number of feature points is as small as possible while the representation capability of the points is as high as possible. The sub-sampled positions of the feature points are determined by a mask generated from a set of training images by means of principal component analysis (PCA). With the feature vector of reduced dimension, a subspace LDA is applied for face recognition, i.e., PCA is first used to reduce the dimension of the Gabor feature vectors generated from the sub-sampled positions, and then a common LDA is applied. Experimental results show that the new sampling method is simple, and effective for both dimension reduction and image representation. The recognition rate based on our proposed scheme is also higher than that achieved using a regular sampling method in a face region. [Copyright &y& Elsevier]

Hausdorff distance is an efficient measure of the similarity of two point sets. In this paper, we propose a new spatially weighted Hausdorff distance measure for human face recognition. The weighting function used in the Hausdorff distance measure is based on an eigenface, which has a large value at locations of importance facial features and can reflect the face structure more effectively. Two modified Hausdorff distances, namely, “spatially eigen-weighted Hausdorff distance” (SEWHD) and “spatially eigen-weighted ‘doubly’ Hausdorff distance” (SEW2HD) are proposed, which incorporate the information about the location of important facial features such as eyes, mouth, and face contour so that distances at those regions will be emphasized. Experimental results based on a combination of the ORL, MIT, and Yale face databases show that SEW2HD can achieve recognition rates of 83%, 90% and 92% for the first one, the first three and the first five likely matched faces, respectively, while the corresponding recognition rates of SEWHD are 80%, 83% and 88%, respectively. [Copyright &y& Elsevier]

The skeleton is essential for general shape representation. The commonly required properties of a skeletonization algorithm are that the extracted skeleton should be accurate; robust to noise, position and rotation; able to reconstruct the original object; and able to produce a connected skeleton in order to preserve its topological and hierarchical properties. However, the use of a discrete image presents a lot of problems that may influence the extraction of the skeleton. Moreover, most of the methods are memory-intensive and computationally intensive, and require a complex data structure.In this paper, we propose a fast, efficient and accurate skeletonization method for the extraction of a well-connected Euclidean skeleton based on a signed sequential Euclidean distance map. A connectivity criterion is proposed, which can be used to determine whether a given pixel is a skeleton point independently. The criterion is based on a set of point pairs along the object boundary, which are the nearest contour points to the pixel under consideration and its 8 neighbors. Our proposed method generates a connected Euclidean skeleton with a single pixel width without requiring a linking algorithm or iteration process. Experiments show that the runtime of our algorithm is faster than the distance transformation and is linearly proportional to the number of pixels of an image. [Copyright &y& Elsevier]

In this paper, an efficient algorithm for detecting human faces in color images is proposed. The first step of our algorithm is to segment the possible skin-like regions in an image by using color information. One of the major problems of using skin color is that a face region may not be detected under poor lighting conditions, or if the lighting conditions vary over the face region. Our approach considers the distributions of the color components of skin pixels under different illuminations. This information can be used to identify skin-color pixels reliably under varying lighting conditions. The skin-color regions are then clustered and verified as human face regions. In order to improve the reliability and accuracy, an eigenmask that has a large magnitude at the important facial features of a human face is used in the detection. Experimental results show that this algorithm can detect human faces under varying lighting conditions reliably and fast. [Copyright &y& Elsevier]

The edge map of a facial image contains abundant information about its shape and structure, which is useful for face recognition. To compare edge images, Hausdorff distance is an efficient measure that can determine the degree of their resemblance, and does not require a knowledge of correspondence among those points in the two edge maps. In this paper, a new modified Hausdorff distance measure is proposed, which has a better discriminant power. As different facial regions have different degrees of significance for face recognition, a new modified Hausdorff distance is proposed which is weighted according to a weighted function derived from the spatial information of the human face; hence crucial regions are emphasized for face identification. Experimental results show that the distance measure can achieve recognition rates of 80%, 87%, and 91% for the first, the first five, and the first seven likely matched faces, respectively. [Copyright &y& Elsevier]