Automatic Facial Expression Recognition Using Modified LPQ and HOG Features with Stacked Deep Convolutional Autoencoders

The fusion of discriminative features and deep autoencoders has received limited attention in the facial expression classification problem. The proposed work aims to improve the classification accuracy of facial expression recognition systems by deriving the abstract, robust, and highly discriminative feature space. The salient contributions of the work are, developing a novel and hybrid distribution by integrating the discriminative information derived from the modified local phase quantization feature and histogram of oriented gradients feature; multi-stage stacked deep convolutional autoencoder (SDCA) is reported to extract the expression-specific latent and highly discriminative feature representation. The analysis of the proposed work is carried out in two phases. In the first phase, the feature representation is derived from the whole face irrespective of the contribution of facial regions to the expression of interest (Holistic approach). The facial regions containing highly discriminative feature for expression classification are identified and features are derived only from salient regions in the second phase (Component-based approach). The work is implemented on the benchmark datasets CK + , KDEF, and JAFFE. The misclassification rates can be effectively reduced by stacking more layers into the SDCA network. Extensive testing reveals that the method achieved a remarkable classification accuracy of 96.7% on CK + , 94.7% on KDEF, and 93.5% on the JAFFE datasets. The empirical results support the effectiveness of the proposed model, demonstrating its improved accuracy in expression classification on benchmark datasets.