Smart Affect Recognition System for Real-Time Biometric Surveillance Using Hybrid Features and Multilayered Binary Structured Support Vector Machine

Human affect recognition (HAR) using images of facial expression and electrocardiogram (ECG) signal plays an important role in predicting human intention. This system improves the performance of the system in applications like the security system, learning technologies and health care systems. The primary goal of our work is to recognize individual affect states automatically using the multilayered binary structured support vector machine (MBSVM), which efficiently classify the input into one of the four affect classes, relax, happy, sad and angry. The classification is performed efficiently by designing an efficient support vector machine (SVM) classifier in multilayer mode operation. The classifier is trained using the 8-fold cross-validation method, which improves the learning of the classifier, thus increasing its efficiency. The classification and recognition accuracy is enhanced and also overcomes the drawback of ‘facial mimicry’ by using hybrid features that are extracted from both facial images (visual elements) and physiological signal ECG (signal features). The reliability of the input database is improved by acquiring the face images and ECG signals experimentally and by inducing emotions through image stimuli. The performance of the affect recognition system is evaluated using the confusion matrix, obtaining the classification accuracy of 96.88%.