Dynamic facial expression analysis based on extended spatio-temporal histogram of oriented gradients

Facial expression is crucial for proper analysis of a person’s face. It is an indicator of the emotion of a person and thus has attracted the attention of many researchers. In this work, a novel local spatio-temporal descriptor is proposed for motion pattern detection. The proposed feature comprises histogram of 3D gradients and the gradients’ variation over time to robustly describe the spatial and temporal information. It also incorporates spatio-temporal pyramid structure to handle different resolution and frame rate. To reduce the dimension of the feature, we applied genetic algorithm for region-based feature selection. We evaluated the performance of our proposed descriptors on facial expression recognition using the Cohn-Kanade (CK ⁺ ) database. The experimental results achieved 96.10% accuracy in detecting six basic emotions. The key advantages of our proposed method are: local and dynamic processing, simple implementation, high performance, and robustness to variation of video resolution or temporal sampling rate.