Search

In this paper, we present a unified representation based on the spatio-temporal steerable pyramid (STSP) for the holistic representation of human actions. A video sequence is viewed as a spatio-temporal volume preserving all the appearance and motion information of an action in it. By decomposing the spatio-temporal volumes into band-passed sub-volumes, the spatio-temporal Laplacian pyramid provides an effective technique for multi-scale analysis of video sequences, and spatio-temporal patterns with different scales could be well localized and captured. To efficiently explore the underlying local spatio-temporal orientation structures at multiple scales, a bank of three-dimensional separable steerable filters are conducted on each of the sub-volume from the Laplacian pyramid. The outputs of the quadrature pair of steerable filters are squared and summed to yield a more robust oriented energy representation. To be further invariant and compact, a spatio-temporal max pooling operation is performed between responses of the filtering at adjacent scales and over spatio-temporal neighbourhoods. In order to capture the appearance, local geometric structure and motion of an action, we apply the STSP on the intensity, 3D gradients and optical flow of video sequences, yielding a unified holistic representation of human actions. Taking advantage of multi-scale, multi-orientation analysis and feature pooling, STSP produces a compact but informative and invariant representation of human actions. We conduct extensive experiments on the KTH, UCF Sports and HMDB51 datasets, which shows the unified STSP achieves comparable results with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Feature analysis plays an important role in many multispectral image applications and scale invariant feature transform (SIFT) has been successfully applied for extraction of image features. However, the existing SIFT algorithms cannot extract features from multispectral images directly. This paper puts forward a novel algorithmic framework based on the SIFT for multispectral images. Firstly, with the theory of the geometric algebra (GA), a new representation of multispectral image including spatial and spectral information is put forward and discussed. Secondly, a new method for obtaining the scale space of the multispectral image is proposed. Thirdly, following the procedures of the SIFT, the GA based difference of Gaussian images are computed and the keypoints can be detected in the GA space. Fourthly, the feature points are finally detected and described in the mathematical framework of the GA. Finally, the comparison results show that the GA-SIFT outperforms some previously reported SIFT algorithms in the feature extraction from a multispectral image, and it is comparable with its counterparts in the feature extraction of color images, indicating good performance in various applications of image analysis. [ABSTRACT FROM AUTHOR]