Search

Video-based human recognition at a distance remains a challenging problem for the fusion of multi-modal biometrics. As compared to the approach based on match score level fusion (Chap. 10), in this chapter, we present an approach that utilizes and integrates information from side face and gait at the feature level. The features of face and gait are obtained separately using principal component analysis (PCA) from enhanced side face image (ESFI) and gait energy image (GEI), respectively. Multiple discriminant analysis (MDA) is employed on the concatenated features of face and gait to obtain discriminating synthetic features. This process allows the generation of better features and reduces the curse of dimensionality. The proposed scheme is tested using two comparative data sets to show the effect of changing clothes and face changing over time. Moreover, the proposed feature level fusion is compared with the match score level fusion and another feature level fusion scheme. The experimental results demonstrate that the synthetic features, encoding both side face and gait information, carry more discriminating power than the individual biometrics features, and the proposed feature level fusion scheme outperforms the match score level and another feature level fusion scheme. The performance of different fusion schemes is also shown as cumulative match characteristic (CMC) curves. They further demonstrate the strength of the proposed fusion scheme. [ABSTRACT FROM AUTHOR]

This book has focused on human recognition at a distance by integrating gait and face in video. The research has demonstrated that the proposed video-based fusion system is effective for human identification. The representation of face and gait, where both fuse information from multiple video frames, is promising in real-world applications. The integration of face and gait biometrics will be highly useful in practical applications. Several important problems are addressed in this book. A summary of key contributions in gait-based human recognition, video-based face recognition and fusion of gait and face for individual recognition is given in this chapter. [ABSTRACT FROM AUTHOR]

This chapter introduces a video based recognition method to recognize non-cooperating individuals at a distance in video, who expose side views to the camera. Information from two biometric sources, side face and gait, is utilized and integrated for recognition. For side face, an enhanced side face image (ESFI), a higher resolution image compared with the image directly obtained from a single video frame, is constructed, which integrates face information from multiple video frames. For gait, the gait energy image (GEI), a spatio-temporal compact representation of gait in video, is used to characterize human walking properties. The features of face and gait are obtained separately using the principal component analysis (PCA) and the multiple discriminant analysis (MDA) combined method from ESFI and GEI, respectively. They are then integrated at the match score level by using different fusion strategies. The approach is tested on a database of video sequences, corresponding to 45 people, which are collected over seven months. The different fusion methods are compared and analyzed. The experimental results show that (a) better face features are extracted from ESFI compared to those from the original side face images; (b) the synchronization of face and gait is not necessary for face template ESFI and gait template GEI. The synthetic match scores combine information from them; and (c) integrated information from side face and gait is effective for human recognition in video. [ABSTRACT FROM AUTHOR]

Human recognition from arbitrary views is an important task for many applications, such as visual surveillance, covert security and access control. It has been found to be difficult in reality, especially when a person is walking at a distance in real-world outdoor conditions. For optimal performance, the system should use as much information as possible from the observations. In this chapter, we introduce a video based system which combines cues of face profile and gait silhouette from the single camera video sequences. It is difficult to get reliable face profile information directly from a low-resolution video frame because of limited resolution. To overcome this problem, we first construct a high-resolution face profile image from multiple adjacent low-resolution frames for each video sequence. Then, we extract face features from the high-resolution profile image. Finally, dynamic time warping (DTW) is used as the matching method to compute the similarity of two face profiles based on the absolute values of curvature. For gait, we use gait energy image (GEI), a spatio-temporal compact representation, to characterize human walking properties. Gait recognition is carried out based on the direct GEI matching. Several schemes are considered for fusion of face profile and gait. A number of dynamic video sequences are tested to evaluate the performance of our system. Experimental results are compared and discussed. [ABSTRACT FROM AUTHOR]

The widespread use of super-resolution methods in a variety of applications such as surveillance has led to an increasing need for quality assessment measures. The current quality measures aim to compare different fusion methods by assessing the quality of the fused images. They consider the information transferred between the super-resolved image and input images only. In this chapter, we propose an integrated objective quality evaluation measure for super-resolved images, which focuses on evaluating the quality of super-resolved images that are constructed from different conditions of input images. The proposed quality evaluation measure combines both the relationship between the super-resolved image and the input images, and the relationship between the input images. Using the proposed measure, the quality of the super-resolved face images constructed from videos are evaluated under different input conditions, including the variation of pose, lighting, facial expressions and the number of input images. [ABSTRACT FROM AUTHOR]

In this chapter, we address the problem of super-resolution of facial images in videos that are acquired at a distance. In particular, we consider (a) a closed-loop approach for super-resolution of frontal faces, (b) super-resolution of frontal faces with facial expressions, and (c) super-resolution of side face images. The details of these technical approaches and experimental results are presented in this chapter. [ABSTRACT FROM AUTHOR]

In this chapter, we approach the task of human silhouette extraction from color and thermal image sequences using automatic image registration. Image registration between color and thermal images is a challenging problem due to the difficulties associated with finding correspondence. However, moving people in a static scene provide cues to address this problem. We first propose a hierarchical scheme to automatically find the correspondence between the preliminary human silhouettes extracted from synchronous color and thermal image sequences for image registration. Next, we discuss strategies for probabilistically combining cues from registered color and thermal images for improved human silhouette detection. It is shown that the proposed approach achieves good results for image registration and human silhouette extraction. Experimental results also show a comparison of various sensor fusion strategies and demonstrate the improvement in performance over non-fused cases for human silhouette extraction. [ABSTRACT FROM AUTHOR]

In this chapter, we propose a kinematic-based approach to recognize individuals by gait using a single camera or multiple cameras for 2D gait recognition where the model is in 3D and the data are in 2D. In addition, we present a 3D gait recognition approach where both the model and the gait data are in 3D. Detailed technical approaches and experimental results are presented for both 2D and 3D gait recognition. [ABSTRACT FROM AUTHOR]

Gait has been evaluated as a new biometric through psychological experiments. However, most gait recognition approaches do not give their theoretical or experimental performance predictions. Therefore, the discriminating power of gait as a feature for human recognition cannot be evaluated. In this chapter, a Bayesian based statistical analysis is performed to evaluate the discriminating power of static gait features (body part dimensions). Through probabilistic simulation, we not only predict the probability of correct recognition (PCR) with regard to different within-class feature variance, but also obtain the upper bound on PCR with regard to different human silhouette resolution. In addition, the maximum number of people in a database is obtained given the allowable error rate. This is extremely important for gait recognition in large databases. [ABSTRACT FROM AUTHOR]

In this chapter, the GEI-based general framework presented earlier is used for individual recognition in diverse scenarios. Insufficient training data associated with an individual is a major problem in gait recognition due to the difficulty of the data acquisition. To address this issue, we not only compute real templates from training silhouette sequences directly, but also generate synthetic templates from training sequences by simulating silhouette distortion. Features learned from real templates characterize human walking properties provided in training sequences, and features learned from synthetic templates predict gait properties under other conditions. A feature fusion strategy is therefore applied at the decision level to improve recognition performance. [ABSTRACT FROM AUTHOR]

In this chapter, we first present a spatio-temporal gait representation, called gait energy image (GEI), to characterize human walking properties. Next, a general GEI-based framework is developed to deal with human motion analysis under different situations. The applications of this general framework will be discussed in the next chapter. [ABSTRACT FROM AUTHOR]

This book addresses the problem of recognizing people at a distance in video. Recognizing humans and their activities have become important research topics in image processing, computer vision and pattern recognition. Related research in biometrics is placed at high priority for homeland security, border control, anti-terrorism, and automated surveillance since it integrates identity information with these tasks. [ABSTRACT FROM AUTHOR]