Search

Multi-human tracking in the crowded environment is a challenging problem due to occlusions, pose change, viewpoint variation and cluttered background. In this work, we propose a robust feature learning for tracking-by-detection methods based on second-order attention network that can capture higher-order relationships between salient features at the early stages of Convolutional Neural Network (CNN). Guided Second-Order Attention Network (GSAN) that, unlike the existing attention learning methods which are weakly-supervised, uses a supervisory signal based on the quality of the self-learned attention maps. More specifically, GSAN looks into the attended maps of a person having the highest confidence and supervise itself to look into the correct regions in the images of the person. Attention maps learned this way are spatially aligned and thus robust to camera-view changes and body pose variations. We verify the effectiveness of our approach by comparing with the state-of-the-art methods on challenging person re-identification and multi object tracking (MOT) datasets.

A new approach for autonomous exploration in an unknown scenario based on the concept of frontiers is proposed in this paper. Exploration frontiers introduced by [4], are the regions on the boundary between open space and unexplored space. A mobile robot is able to construct its map by adding new space and moving to unvisited frontiers until the entire environment has been explored. However, the original frontier strategy, suffering from local minima, only considers distance and size of unknown spaces, resulting in low exploration efficiency in complex environments. By making use of the robot heading information using wheel odometry and coarse graph representation of the environment, the modified exploration method is able to balance the mapping coverage and time expenditure to a greater extent. The proposed method is experimentally verified on a mobile platform, exploring a real-world office environment cluttered with a variety of obstacles.

In recent years, an attempt is being made to control robots more intuitive and intelligible by exploiting and integrating anthropomorphic features to boost social human-robot interaction. The design and construction of anthropomorphic robots for this kind of interaction is not the only challenging issue -- smooth and expectation-matching motion control is still an unsolved topic. In this work we present a highly configurable, portable, and open control framework that facilitates anthropomorphic motion generation for humanoid robot heads by enhancing state-of-the-art neck-eye coordination with human-like eyelid saccades and animation. On top of that, the presented framework supports dynamic neck offset angles that allow animation overlays and changes in alignment to the robots communication partner whileretaining visual focus on a given target. In order to demonstrate the universal applicability of the proposed ideas we used this framework to control the Flobi and the iCub robot head, both in simulation and on the physical robot. In order to foster further comparative studies of different robot heads, we will release all software, based on this contribution, under an open-source license.

This paper presents an approach to guide a robot arm visually, by using an active stereo camera mounted on an active head platform. The camera and head system is capable of changing the look-direction and fixating on the target object. Unlike the traditional AI approach, the proposed approach does not require full perspective camera calibration or scene reconstruction. Visual parameters which contain relationships with the robot or world space are used instead of the reconstructed three-dimensional (3D) world information. The necessary visual cues required to control the robot are developed. These visual cues are computed solely from the image coordinates. The relationship between these visual cues and the robot space is then established via a mapping matrix. Using this relation together with visual feedback, the end-effector can be guided towards the target object. Even though the 3D world coordinates are not recovered, the positioning accuracy attainable is high. To allow for vergence movement of the stereo camera or pan–tilt movements of the head platform, methods to compensate the mapping matrix are proposed. This approach eliminates the need for re-calibration after any deliberate change in the configuration of the head platform. Therefore, building an autonomous robot with coordinated head, eye and hand becomes technically feasible. Simulation studies on the capability of the system to reach arbitrary target objects are presented. High flexibility and large accessible workspace are the main advantages of the proposed autonomous robot system.

The conventional mouth gender recognition is based on a static image and ignore the dynamic information. In this paper, we propose a lip movement gender recognition method to improve the accuracy by exploring the dynamic information while a user is speaking. In order to overcome the difficulty caused by the nonlinear distribution of the lip images, Gausian Mixture Models (GMM) is adopted to represent the lip images. A similarity measure is defined to measure the difference between the successive frames. Gender recognition, as a soft biometric trait, can provide useful information for improving the performance of the speaker recognition systems. The accuracy of voice-based speaker gender recognition is high if the condition of the environment is good. But it will be drastically decreased if the test is conduted in a noisy environment. In this paper, we showed that lip movement, considered as a sequence of mouth images, can provide additional information than mouth alone for recognizing gender Experimental result obtained showed the effectiveness of the proposed method which is comparable to using just the voice information.

Over the last decade, automated analysis of human affective behavior has become an active research area in computer science, psychology, neuroscience, and related fields. This study investigates the application of Gabor filter based features in combination of Genetic Algorithm (GA) and Support Vector Machine (SVM) for dynamic analysis of six basic facial expressions from video sequences. Traditionally, a set of Gabor filters is used for feature extraction from static images of face. However, we employed Sum of Difference (SOD) approach to analysis the dynamics of facial expression from a video sequence. We also used GA to overcome the problem of high dimensional feature vectors and computation cost. A local Gabor filter bank with selected frequencies and orientations is produced by GA. The experimental results show that the proposed method is effective for temporal analysis of affective states. The detection rate of six basic emotions has been reached to 92.97% for Cohn-Kanade (CK+) database. ASTAR (Agency for Sci., Tech. and Research, S’pore) Accepted version

There has been increasing interest on the use of video surveillance to analyze human biometrics, especially in places where the chance of occurring crime is high, such as in large cities. As an example, 2185 children are being reported missing daily in the United States. It has been rarely looked into belonging a person to the same family as a soft biometric modal. We term the ability to verify query images as family members “family (kinship) verification.” Several studies have been conducted on family photo albuming, but the problem of family verification has not yet been directly addressed and analyzed. The lack of comprehensive published family datasets is evidence that the matter has received very little attention to date. This thesis focuses on providing solutions for challenges of family verification as a recent application of computer vision. We propose frameworks to recognize members of the same family based on the facts and psychological studies on family members’ resemblance to each other. We also investigate the demanding case of missing/unknown member verification as a critical real-life application that will benefit from this research. In the first major section of this thesis, we study true assumptions behind the resemblance of family members to one another, and the challenges of family verification. We develop traditional face recognition approaches to facilitate family verification by extraction of facial resemblance among family members. We investigate the very recent family verification problem, define scenarios and collect a dataset of family albums. Our aim is to recognize if the query sample belongs to the same family. In the first stage, we investigate facial resemblance extraction among family members to perform family verification. Similarities of facial patches are utilized as multiple experts to recognize family members. Finally, our proposed inter-patch constraint-free algorithm of facial patches analysis determines the redundant patches among family members. On average, five facial patches (29% of the facial patches) are chosen to achieve the same accuracy obtained by using all patches for family verification. Consequently, the computational time is saved by 71% compared to utilizing all patches. The second major section of our thesis is focused on family verification in real-time systems to obtain more discriminative features with the least dimension. Firstly, we minimize the quantization error of the Local Binary Pattern (LBP) operator through the incorporation of uniformly-sampled thresholds for LBP (UTLBP). Up to 3.4% improvement was achieved for family verification compared to various features in the literature, while the feature dimension is considerably smaller than the well-known Haar-like features (60%). Moreover, the proposed operator is more robust than the conventional operators, as tested on the large CAS-PEAL dataset against background, illumination, aging, and accessories changes. It achieves up to 8% performance improvement depending on the CAS-PEAL probe set, compared to the best operator in the literature. Then, we propose a novel redundant feature removal algorithm based on the fact that there is less similarity among the faces of family members than that of an individual’s face samples. The proposed algorithms are then employed on the UTLBP feature operator to select the most informative set of thresholds to outperform the Haar-like features, with 20 times less feature dimension. The proposed approach converges 10 times faster than the state-of-the-art face recognition algorithms do. The third contribution of the thesis is to utilize from each member’s image segment resemblance to the entire family. Our proposed method is to incorporate the degree of resemblance of each individual member’s patch to perform family verification. Our analysis and the important consensuses of psychologists’ studies reveal that the facial resemblance between family members differs from member to member and is facial patch specific. We propose to estimate the amount of resemblance for each patch based on the prior information of the member. As the score level fusion is preferred in the literature due to its wealth of information and low data dimension, we embed the available a priori information in the score fusion rule. The proposed method outperforms the state-of-the-art score fusion rules in all scenarios by 14%. Moreover, the resemblance estimation slightly outperforms human observation in the designated survey. The proposed method could also improve object family verification on the selected Caltech-256 database by 8%. In summary, two automated frameworks for recognizing family members are proposed. The first framework utilizes UTLBP operator and the novel redundant feature removal provides fast and low computational algorithms suitable for real-time processing. The other framework of facial patch resemblance extraction among family members benefits from offline training with parallel computing capability. Finally, the enhancement of family verification can potentially solve critical applications of daily life to find missing children and unknown parents. DOCTOR OF PHILOSOPHY (EEE)

In recent decades, automatic fingerprint identification system has attracted significant interest among the scientific research community. Though large amount of effort has been made in both the academic and industrial communities, there are still unsolved problems. Noisy image processing and large scale database processing are two typical problems. Fingerprint images consist of oriented texture. Therefore the orientation information plays a very important role in fingerprint processing. Moreover, the orientation of the fingerprint follows a certain structure that is not random. Such oriented structure provides a possibility to recover the orientation information in fingerprints corrupted by noise, even for a large noise patch which cannot be solved by the traditional gradient based methods. Accurate reconstruction of the orientations is useful for fingerprint filtering, segmentation, classification and recognition. Several methods have been proposed to reconstruct the fingerprint orientation. In this thesis, phase portrait model approach is used. The first order phase portrait is demonstrated to be robust for both occluded and noisy fingerprints. It also has the capability to describe the orientation patterns of the core point and delta point in a fingerprint image. Therefore a method based on the first order phase portrait is proposed to reconstruct the local orientation near the core and delta points. Furthermore, considering the regularity of the fingerprint orientation and linearity of the first order phase portrait, a model based on the piecewise phase portrait is used to predict the orientations far away from the singular point region. The advantage of this prediction model is that it provides a way to overcome the large patch of noise or missing orientation. To obtain the fingerprint orientation with higher precision, more complicated model based on the constrained nonlinear phase portrait approach is proposed. The model employs the advantages of both the first order and high order phase portrait model. Hence accurate reconstructed orientations at both the local and global area can be obtained simultaneously. This thesis also addresses fingerprint classification using the coefficients of the constrained nonlinear phase portrait model together with the singularities information. Since the coefficients of the constrained nonlinear phase portrait model play a key role in reconstructing the fingerprint orientation, it is also expected to be able to effectively classify fingerprints. A fingerprint classification algorithm using the orientation model and the singular points' information which complement the orientation information is developed and is shown to have good classification performance. DOCTOR OF PHILOSOPHY (EEE)

Face detection, localization and recognition from multi-poses are one of the most challenging topics in the area of computer vision and pattern recognition. During the past decade several methods have been proposed for face detection and recognition but they mainly concentrate on frontal faces. Although in recent years, some methods have been proposed based on sliding windows to exhaustively search through the image in many scales and views, they are slow and their performance on multi-view or multi-pose faces are not as good as frontal face detections. Training these methods is also a bottleneck. Most of the proposed methods require thousands of positive and negative samples from different face poses for training. These images for training have to be manually cropped, aligned and labeled, thus is manually intensive and expensive to perform. Note that since the effect of variation in the face pose captured on a fix camera and change in camera view but with a fixed face pose is the same, we used both words interchangeably, In this research project, a probabilistic approach is proposed for face detection, localization and identification from multi-views. The proposed method does not supervision during the training stage. The main focus of the project is on multi-view analysis as well as unsupervised or automated learning. The proposed approach provides a unified framework to learn a multi-view model for face detection, localization and identification. For face detection and localization from multi poses, given the images of different people in multi-poses, it obtains a multi-view face model obtained through a constellation of corresponding face features in the training set. The obtained model is pruned such that only the most distinctive features are retained. The model is then used to detect and localize multiple face images in multi-poses. Two versions of model construction are proposed for face detection and localization. One requires manually labeling only two control points in one image of the training set regardless of the number of images in the training set. The other version is completely automatic. Even the control points are estimated automatically and the system only requires some training images that include face of different people from multi-views. The trade-off is that the accuracy is slightly lower compared to the manually labeled approach. Another effort is face identification and localization from multi poses. The proposed method is completely automatic. The data collection is also performed by the system using web images obtained from web search. It requires only the textual name of the queried person, which is used as input to the search engine. Using the images returned by the search engine, the system classifies these images and constructs a multi-view face model for the query. With the model, the system is then able to identify the query candidate in his/her digital photo album or gallery. The user does not have to manually tag the images or provide images to train the face recognition module. Since system is completely automatic, it has widely application including search for people in news video or tagging actors and actresses in movies or targeted advertisement for interactive televisions (IPTV). DOCTOR OF PHILOSOPHY (EEE)

Gaussian process (GP) is a stochastic process that has been studied for a long time and gained wide interests in the machine learning community in recent years. In this thesis, several interesting pattern analysis problems are solved using Gaussian process. Gaussian process models can be interpreted in two views, the weight space view and the function space view. Their different interpretations can be helpful in applying GPs to solve real problems. Its capability as functional prior inspired the original work on learning nonparametric similarity measure in this thesis. The similarity between pairwise inputs is considered a smooth function described by a GP prior. Given known similarity constraints, the model can be tuned by maximum likelihood and similarity on new inputs can be inferred. The advantage is that the learned similarity measure does not assume any parametric form. It is flexible to model various data with arbitrary distributions, and handles noise and outliers in the data. Gaussian process latent variable model (GPLVM) is a data modeling tool that is derived based on Gaussian processes from the weight space view. It was originally proposed for visualization of high dimensional data. DOCTOR OF PHILOSOPHY (EEE)