Facial emotion recognition has garnered significant interest in recent years due to its wide-ranging applications in human-computer interaction, affective computing, and psychological research. In this paper, a capsule neural network-based model is proposed which incorporates an attention mechanism to improve the classification performance. Specifically, an empirical investigation into the effectiveness of various attention mechanisms in enhancing facial emotion recognition systems is presented. While looking in that direction, four attention mechanisms are explored with their comparison: channel attention, spatial attention, CBAM attention, self-attention, and multi-head attention (MHA) attention. An experiment has been conducted using CapsuleNet architecture on an in-the-wild facial emotion recognition dataset. The results demonstrate the distinct impacts of each attention mechanism on improving the recognition performance for different facial expressions. Furthermore, the computational complexity and interpretability of these attention mechanisms are analyzed to provide insights into their practical feasibility. This research contributes to the understanding of attention mechanisms in the context of facial emotion recognition. It offers valuable guidance for the design and optimization of emotion recognition systems in real-world applications.