Search

Multi-human tracking in the crowded environment is a challenging problem due to occlusions, pose change, viewpoint variation and cluttered background. In this work, we propose a robust feature learning for tracking-by-detection methods based on second-order attention network that can capture higher-order relationships between salient features at the early stages of Convolutional Neural Network (CNN). Guided Second-Order Attention Network (GSAN) that, unlike the existing attention learning methods which are weakly-supervised, uses a supervisory signal based on the quality of the self-learned attention maps. More specifically, GSAN looks into the attended maps of a person having the highest confidence and supervise itself to look into the correct regions in the images of the person. Attention maps learned this way are spatially aligned and thus robust to camera-view changes and body pose variations. We verify the effectiveness of our approach by comparing with the state-of-the-art methods on challenging person re-identification and multi object tracking (MOT) datasets.

A new approach for autonomous exploration in an unknown scenario based on the concept of frontiers is proposed in this paper. Exploration frontiers introduced by [4], are the regions on the boundary between open space and unexplored space. A mobile robot is able to construct its map by adding new space and moving to unvisited frontiers until the entire environment has been explored. However, the original frontier strategy, suffering from local minima, only considers distance and size of unknown spaces, resulting in low exploration efficiency in complex environments. By making use of the robot heading information using wheel odometry and coarse graph representation of the environment, the modified exploration method is able to balance the mapping coverage and time expenditure to a greater extent. The proposed method is experimentally verified on a mobile platform, exploring a real-world office environment cluttered with a variety of obstacles.

Automatically delivering objects to in-car passengers has many potential applications. Such a system generally consists of two sub-systems: a perception system and an action system. The perception system basically looks for the targets' positions and the action system delivers objects to the targets. In this paper, we propose a novel perception system, which contains two major functions: estimation of reaching points and discovering potential risks. The reaching points are the locations where robot arms needs to reach. Moreover, it should be able to reach with comfort by passengers and keep a safe distance from the car body. In order to achieve this, all the vehicle components (side surfaces, side mirrors etc.), which may cause collision, need to be detected. Potential risks are usually caused by moving objects or changing door state (close to open) during the operation. It is necessary to monitor these two situations to avoid any potential risks during operation. Our offline test shows that the accuracy of reaching points estimation can reach up to 94% and the response time for moving objects detection or door state changes is less than 1 millisecond.

This paper presents an approach to guide a robot arm visually, by using an active stereo camera mounted on an active head platform. The camera and head system is capable of changing the look-direction and fixating on the target object. Unlike the traditional AI approach, the proposed approach does not require full perspective camera calibration or scene reconstruction. Visual parameters which contain relationships with the robot or world space are used instead of the reconstructed three-dimensional (3D) world information. The necessary visual cues required to control the robot are developed. These visual cues are computed solely from the image coordinates. The relationship between these visual cues and the robot space is then established via a mapping matrix. Using this relation together with visual feedback, the end-effector can be guided towards the target object. Even though the 3D world coordinates are not recovered, the positioning accuracy attainable is high. To allow for vergence movement of the stereo camera or pan–tilt movements of the head platform, methods to compensate the mapping matrix are proposed. This approach eliminates the need for re-calibration after any deliberate change in the configuration of the head platform. Therefore, building an autonomous robot with coordinated head, eye and hand becomes technically feasible. Simulation studies on the capability of the system to reach arbitrary target objects are presented. High flexibility and large accessible workspace are the main advantages of the proposed autonomous robot system.

The conventional mouth gender recognition is based on a static image and ignore the dynamic information. In this paper, we propose a lip movement gender recognition method to improve the accuracy by exploring the dynamic information while a user is speaking. In order to overcome the difficulty caused by the nonlinear distribution of the lip images, Gausian Mixture Models (GMM) is adopted to represent the lip images. A similarity measure is defined to measure the difference between the successive frames. Gender recognition, as a soft biometric trait, can provide useful information for improving the performance of the speaker recognition systems. The accuracy of voice-based speaker gender recognition is high if the condition of the environment is good. But it will be drastically decreased if the test is conduted in a noisy environment. In this paper, we showed that lip movement, considered as a sequence of mouth images, can provide additional information than mouth alone for recognizing gender Experimental result obtained showed the effectiveness of the proposed method which is comparable to using just the voice information.

Over the last decade, automated analysis of human affective behavior has become an active research area in computer science, psychology, neuroscience, and related fields. This study investigates the application of Gabor filter based features in combination of Genetic Algorithm (GA) and Support Vector Machine (SVM) for dynamic analysis of six basic facial expressions from video sequences. Traditionally, a set of Gabor filters is used for feature extraction from static images of face. However, we employed Sum of Difference (SOD) approach to analysis the dynamics of facial expression from a video sequence. We also used GA to overcome the problem of high dimensional feature vectors and computation cost. A local Gabor filter bank with selected frequencies and orientations is produced by GA. The experimental results show that the proposed method is effective for temporal analysis of affective states. The detection rate of six basic emotions has been reached to 92.97% for Cohn-Kanade (CK+) database. ASTAR (Agency for Sci., Tech. and Research, S’pore) Accepted version

There has been increasing interest on the use of video surveillance to analyze human biometrics, especially in places where the chance of occurring crime is high, such as in large cities. As an example, 2185 children are being reported missing daily in the United States. It has been rarely looked into belonging a person to the same family as a soft biometric modal. We term the ability to verify query images as family members “family (kinship) verification.” Several studies have been conducted on family photo albuming, but the problem of family verification has not yet been directly addressed and analyzed. The lack of comprehensive published family datasets is evidence that the matter has received very little attention to date. This thesis focuses on providing solutions for challenges of family verification as a recent application of computer vision. We propose frameworks to recognize members of the same family based on the facts and psychological studies on family members’ resemblance to each other. We also investigate the demanding case of missing/unknown member verification as a critical real-life application that will benefit from this research. In the first major section of this thesis, we study true assumptions behind the resemblance of family members to one another, and the challenges of family verification. We develop traditional face recognition approaches to facilitate family verification by extraction of facial resemblance among family members. We investigate the very recent family verification problem, define scenarios and collect a dataset of family albums. Our aim is to recognize if the query sample belongs to the same family. In the first stage, we investigate facial resemblance extraction among family members to perform family verification. Similarities of facial patches are utilized as multiple experts to recognize family members. Finally, our proposed inter-patch constraint-free algorithm of facial patches analysis determines the redundant patches among family members. On average, five facial patches (29% of the facial patches) are chosen to achieve the same accuracy obtained by using all patches for family verification. Consequently, the computational time is saved by 71% compared to utilizing all patches. The second major section of our thesis is focused on family verification in real-time systems to obtain more discriminative features with the least dimension. Firstly, we minimize the quantization error of the Local Binary Pattern (LBP) operator through the incorporation of uniformly-sampled thresholds for LBP (UTLBP). Up to 3.4% improvement was achieved for family verification compared to various features in the literature, while the feature dimension is considerably smaller than the well-known Haar-like features (60%). Moreover, the proposed operator is more robust than the conventional operators, as tested on the large CAS-PEAL dataset against background, illumination, aging, and accessories changes. It achieves up to 8% performance improvement depending on the CAS-PEAL probe set, compared to the best operator in the literature. Then, we propose a novel redundant feature removal algorithm based on the fact that there is less similarity among the faces of family members than that of an individual’s face samples. The proposed algorithms are then employed on the UTLBP feature operator to select the most informative set of thresholds to outperform the Haar-like features, with 20 times less feature dimension. The proposed approach converges 10 times faster than the state-of-the-art face recognition algorithms do. The third contribution of the thesis is to utilize from each member’s image segment resemblance to the entire family. Our proposed method is to incorporate the degree of resemblance of each individual member’s patch to perform family verification. Our analysis and the important consensuses of psychologists’ studies reveal that the facial resemblance between family members differs from member to member and is facial patch specific. We propose to estimate the amount of resemblance for each patch based on the prior information of the member. As the score level fusion is preferred in the literature due to its wealth of information and low data dimension, we embed the available a priori information in the score fusion rule. The proposed method outperforms the state-of-the-art score fusion rules in all scenarios by 14%. Moreover, the resemblance estimation slightly outperforms human observation in the designated survey. The proposed method could also improve object family verification on the selected Caltech-256 database by 8%. In summary, two automated frameworks for recognizing family members are proposed. The first framework utilizes UTLBP operator and the novel redundant feature removal provides fast and low computational algorithms suitable for real-time processing. The other framework of facial patch resemblance extraction among family members benefits from offline training with parallel computing capability. Finally, the enhancement of family verification can potentially solve critical applications of daily life to find missing children and unknown parents. DOCTOR OF PHILOSOPHY (EEE)

In recent decades, automatic fingerprint identification system has attracted significant interest among the scientific research community. Though large amount of effort has been made in both the academic and industrial communities, there are still unsolved problems. Noisy image processing and large scale database processing are two typical problems. Fingerprint images consist of oriented texture. Therefore the orientation information plays a very important role in fingerprint processing. Moreover, the orientation of the fingerprint follows a certain structure that is not random. Such oriented structure provides a possibility to recover the orientation information in fingerprints corrupted by noise, even for a large noise patch which cannot be solved by the traditional gradient based methods. Accurate reconstruction of the orientations is useful for fingerprint filtering, segmentation, classification and recognition. Several methods have been proposed to reconstruct the fingerprint orientation. In this thesis, phase portrait model approach is used. The first order phase portrait is demonstrated to be robust for both occluded and noisy fingerprints. It also has the capability to describe the orientation patterns of the core point and delta point in a fingerprint image. Therefore a method based on the first order phase portrait is proposed to reconstruct the local orientation near the core and delta points. Furthermore, considering the regularity of the fingerprint orientation and linearity of the first order phase portrait, a model based on the piecewise phase portrait is used to predict the orientations far away from the singular point region. The advantage of this prediction model is that it provides a way to overcome the large patch of noise or missing orientation. To obtain the fingerprint orientation with higher precision, more complicated model based on the constrained nonlinear phase portrait approach is proposed. The model employs the advantages of both the first order and high order phase portrait model. Hence accurate reconstructed orientations at both the local and global area can be obtained simultaneously. This thesis also addresses fingerprint classification using the coefficients of the constrained nonlinear phase portrait model together with the singularities information. Since the coefficients of the constrained nonlinear phase portrait model play a key role in reconstructing the fingerprint orientation, it is also expected to be able to effectively classify fingerprints. A fingerprint classification algorithm using the orientation model and the singular points' information which complement the orientation information is developed and is shown to have good classification performance. DOCTOR OF PHILOSOPHY (EEE)

Estimating human poses from 2D images or video sequences can provide the moving trajectories of the body joints for the high level processing, human activity recognition, which is applicable in surveillance, human-computer interaction and clinical and sport analysis. This project proposes a new statistical formulation called the data-driven Monte Carlo hidden Markov model to estimate human poses from random initializations. MASTER OF ENGINEERING (EEE)