Search

In this work, we discuss the feature fusion approach of multisurface skeleton projection images for action recognition. Multisurface skeleton projection images are generated from human skeleton joint motion trajectories on three surfaces: horizontal-vertical, horizontal- and vertical-depth surfaces. The vision features of these skeleton projection images contain complementary action information on different surfaces and are generally combined by early fusion or late fusion. To learn and fuse the discriminative features of each surface effectively, this paper proposes a new feature fusion method called the discriminative feature fusion network, which uses a two-task framework to recognize action and surface categories simultaneously. In the proposed network, the features of three skeleton projection images are computed by the same convolutional network. To retain the complementary action feature of these skeleton projection images, the surface classification loss is defined and added into the action classification loss to train the feature learning network. The experimental results show that the performance of the proposed feature fusion method is better than traditional early and late fusion. Compared with skeleton visualization image-based action recognition methods, the proposed method achieves state-of-art performance on the NTU RGB+D action dataset.

Most existing methods of motion assessment system used the contact sensor, infrared sensor, and depth sensor, and few works provided the solution of digital camera. To solve this problem, the authors propose a new motion assessment system based on light camera. In this work, the motion assessment was regarded as pattern regression problem of skeleton joint trajectory. Firstly, the system uses the camera to capture the image sequences. The pose estimation method is used to obtain body skeleton from image. Secondly, due to the difference of motion frequency of each person, the length of the image sequences is different and the length of each joint trajectory also will be different. Fourier transform is applied to normalise the trajectory and use the coefficients of Fourier transform as the joint trajectory feature. Finally, the regression model is built to assess the motion quality. Some experimental results and discussion on action video data are used to verify the effectiveness of the system.