Robust in-plane and out-of-plane face detection algorithm using frontal face detector and symmetry extension.

Abstract Face detection plays an important role in many computer vision applications. In recent years, much research has focused on extending the well-established Adaboost face detector algorithm for multi-view face detection. However, detecting in-plane and out-of-plane rotated faces simultaneously is still a challenging task today. In this paper, a very robust multi-view face detection algorithm, with its core functionality based on frontal face detection, is proposed to simultaneously detect in-plane and out-of-plane rotated faces. Moreover, only the training data in the frontal face is needed and we do not require the training data from different in-plane or out-of-plane rotation angles. In the proposed algorithm, first, techniques such as the skin filter and entropy rate superpixels (ERSs) are applied to obtain face candidates. Then, angle compensation and refinement are applied to improve the accuracy of face detection in the in-plane case. Moreover, the symmetry extension technique, i.e., extending the face candidate with its flipped version to create a face similar to the frontal one, is applied to detect out-of-plane faces without the need of training data. Simulations on the FEI, Pointing'04, BaoFace, Group, Utrecht, and WWW datasets demonstrate the proposed algorithm's effectiveness and superior performance as compared to state-of-the-art face detection methods. Graphical abstract Unlabelled Image Highlights • We detect in-plane and out-of-plane rotated faces by only the frontal face detector. • Training data from different in and out-of-plane rotation angles are not required. • Skin filters and ERSs can preserve the faces from different illumination and races. • Angle compensation computes the rotation angle for each rotated face. • Symmetry extension creates pseudo faces to well detect out-of-plane rotated faces. [ABSTRACT FROM AUTHOR]