FL-CapsNet: facial localization augmented capsule network for human emotion recognition

Abstract Emotion is an important aspect of effective human communication, and hence, facial emotion recognition (FER) has become essential in human–computer interaction systems. The automation of FER has been carried out by many researchers using ML/DL techniques. However, the models developed using...
Ausführliche Beschreibung

Abstract Emotion is an important aspect of effective human communication, and hence, facial emotion recognition (FER) has become essential in human–computer interaction systems. The automation of FER has been carried out by many researchers using ML/DL techniques. However, the models developed using convolutional neural networks (CNNs) bagged high recognition accuracies among different FER approaches. Rather than its high performance, CNN has failed to encode different orientation features since the pooling operations used in CNN for feature extraction omit vital information. Due to omitting vital features, the performance will be reduced while recognizing the emotions from facial images that consists of different orientations. Subsequently, to reduce the problems of CNN such as encoding different orientation features and increased training time, Capsule Networks (CapsNet) were developed. CapsNet is capable of storing 8 such features vectors with the incorporation of dynamic routing approaches and squashing in place of pooling operations to mitigate the issue of rotational invariance. Hence in this paper, we proposed CapsNet for FER in order to enhance the accuracy. However, the facial images that consider for training consist of unwanted information that is not essential for FER, delay the convergence and take more iterations for training facial images. Hence, face localization (FL) is proposed to incorporate with CapsNet in our model to eliminate the back ground noise or unwanted information from the facial images for effective training process. The proposed FL-CapsNet is rigorously tested on benchmark datasets such as JAFFE, CK+, and FER2013 to evaluate the generalization of the proposed model, and it is evidenced that FL-CapsNet outperformed the existing CapsNet-based FER models. Ausführliche Beschreibung