Improving face verification using facial marks and deep CNN: IARPA Janus benchmark-A.

Face verification performance by human brain has been shown to be much better than most of the state-of-the-art approaches in computer vision. Performance improvement of automated face recognition (FR) systems that may equal or surpass the human intellect is the key goal of this research. We present our face verification system using facial mark (FM) combined with deep convolutional neural network (DCNN) approach to improve the overall FR accuracy. We propose to use FM (e.g., scars, moles and freckles) for face matching in the wild where the FM detection is performed on mean faces as well as affine aligned normalized facial images. The FR experiments are carried out on IARPA Janus Benchmark-A (IJB-A) dataset which includes real-world unconstrained images from 500 subjects. The IJB-A datasets includes full pose, expression, and illumination variations which are much harder than traditional FERET and Mugshot datasets. We evaluated the average FR performance using a weighted score-level fusion of FM and DCNN based recognition methods. The experimental evaluations on FERET, CFM and Mugshot datasets show higher performances than state-of-the-art FM approaches with 99.23%, 94.64% and 97.86% accuracies in Rank-1 evaluations, respectively. Our FR performance of FM + DCNN (86.46% in TAR, 91.23% in Rank-1, 96.57% in Rank-5, 98.65% in Rank-10) is shown to be higher than state-of-the-art (83.80% in TAR@1%FAR, 90.30% in Rank-1, 96.5% in Rank-5, and 97.7% in Rank-10). Experimental results after fusion of FM + DCNN on IJB-A dataset show 2.66% FR performance improvement from the DCNN only recognition in terms of TAR@1%FAR. • Face verification with soft biometric based matching approach • Unconstrained dataset with larger variations collected from 500 subjects • Evaluations on FERET, CFM, Mugshot, and IJB-A datasets • Performance improvement with Facial mark and VGG-face based fusion matching Sidra Riaz received her BS degree in Telecom Engineering from NUCES-FAST in 2011. She received her MS and PhD degrees in Computer Engineering from Chosun University in 2013 and Sogang University in 2020, respectively. She is currently working as an AI developer in BluePrint Company, Korea. Ms. Sidra Riaz is the recipient of National ICT R&D Merit Scholarship, Global IT Scholarship, and Seoul Honorary Citizenship Awards. Her research interests include deep learning, computer vision, and pattern recognition. Unsang Park received the BS and MS degrees from the Department of Materials Engineering, Hanyang University, South Korea, in 1998 and 2000, respectively. He received the MS and Ph.D. degrees from the Department of Computer Science and Engineering, Michigan State University, in 2004 and 2009, respectively. From 2012, he is an assistant professor in the Department of Computer Science and Engineering at Sogang University. His research interests include pattern recognition, image processing, computer vision, and machine learning. Prem Natarajan is Michael Keston Executive Director of ISI, a vice dean of the USC Viterbi School of Engineering and a professor of computer science. At ISI, he leads managerial and technical directions Institute-wide, including research, development and the MOSIS electronic chip brokerage. He also heads teams in his areas of expertise: novel approaches to face, character, handwriting and speech recognition, along with other deep learning and natural language processing directions. [ABSTRACT FROM AUTHOR]