Search

Hand gesture recognition on the depth videos is a promising approach for automotive interfaces because it is less sensitive to light variation and more accurate than other traditional methods. However, video gestures recognition is still a challenging task since lots of interferences are induced by the uncorrelated gesture factors. Considering that if the displays are more relevant, the results will more accurate, so ResNext, a kind of compact and efficient neural network, is firstly used as feature extractor, then an improved weighted frame unification method is adopted to obtain the key frame samples, finally the Discriminant correlation analysis (DCA) is employed to fuse features for static data and dynamic data after conducting Feature embedding branch (FEB) on static data. The public dataset named Depth based gesture recognition database (DGRD) is used in this paper, but the dataset is a little small and the class distribution is largely imbalance, and we find the performance of ResNext degrades badly in the condition of imbalance problem although it achieves excellent result at sufficient training data. In order to conquer the disadvantages of limited dataset, a special loss function scheme combining the softmax loss and dice loss is proposed. Evaluation of the algorithm performances in comparison with other state-of-the-art methods indicates that the proposed method is more practical for gesture recognition and may be widely adopted by automotive interfaces.

Video prediction is a meaningful task for it has a wide range of application scenarios. And it is also a challenging task since it needs to learn the internal representation of a given video for both appearance and motion dynamics. The existing methods regard this problem as a spatiotemporal sequence forecasting problem and try to resolve it in a one-shot fashion, which causes the prediction result being blurry or inaccurate. So, a more intuitional thought is to split this problem into two parts: model the dynamic pattern of the given video and learn the appearance representation of given video frames. In this paper, we develop a novel network structure named MixPred based on this idea to address this issue. We divide the prediction problem into two parts as mentioned above and build two subnets to solve these two parts separately. Instead of fusing the results of subnets at the final layer, we put forward a parallel interaction style through the whole process to merge the dynamic information and content information in a more natural way. Besides, we propose three different connection methods for exploring the most effective connection structure. We trained the model on UCF-101 and KITTI, and testing our model on UCF-101, KITTI, and Caltech. The results demonstrate that our method achieves state-of-the-art both quantitatively and qualitatively.