Video-based facial expression recognition using learned spatiotemporal pyramid sparse coding features.

Recently, hand-designed local descriptors like spatiotemporal Gabor filters and VLBP have been successfully applied in video-based facial expression recognition. One major drawback of these methods is that they are hard to generalize to different problems. In this paper, we propose a new video-based facial expression recognition method by automatically learning features from video data. Specifically, we use sparse coding algorithm to learn spatiotemporal features from unlabeled facial expression videos. For modeling spatiotemporal layout information embedded in facial expressions to improve recognition performance, we extend the idea of spatial pyramid matching (SPM) into video case, and perform spatiotemporal pyramid feature pooling following sparse coding feature extraction. Experimental results on widely used Cohn–Kanade database show that the classification performance can be improved effectively by considering spatiotemporal layout of facial expressions, and our method outperforms popular methods using hand-designed features. [ABSTRACT FROM AUTHOR]