Here, this paper introduces a work on utilizing technology to recognize different kinds of gestures in human conversations. All the tasks were organized in a database; they followed commands so that they could display eight distinct emotional states. In the system, three ways were used to collect features: recognition of facial expressions, gesture recognition with MediaPipe, and analysis of sound. Since MediaPipe is a machine learning tool for tracking and detecting gestures, it proved to be important for analyzing hand movements. It applied CNNs to detect where the main landmarks of hands are, which made gesture recognition more accurate. After that, the system used a Bayesian classifier to electronically sort the emotions using data. To see the effect, the data were tested using unimodal, bimodal, and multimodal methods. It was done before, or after, the materials were classified. It was shown that the best multimodal fusion achieved more than a 10% increase in recognition compared to the outstanding unimodal system. Out of the tested methods, ‘gesture-acoustic’ was the best. Adding the third type of data led to improvements that were above the achievement of the best bimodal model.