Comparing Hybrid NN-HMM and RNN for Temporal Modeling in Gesture Recognition

This paper provides an extended comparison of two temporal models for gesture recognition, namely Hybrid Neural Network-Hidden Markov Models (NN-HMM) and Recurrent Neural Networks (RNN) which have lately claimed the state-the-art performances. Experiments were conducted on both models in the same body of work, with similar representation learning capacity and comparable computational costs. For both solutions, we have integrated recent contributions to the model architectures and training techniques. We show that, for this task, Hybrid NN-HMM models remain competitive with Recurrent Neural Networks in a standard setting. For both models, we analyze the influence of the training objective function on the final evaluation metric. We further tested the influence of temporal convolution to improve context modeling, a technique which was recently reported to improve the accuracy of gesture recognition.