The primary aim of this research is to enhance sentiment analysis accuracy, particularly for reviews containing both text and emojis. To achieve this, we employed deep learning models such as bidirectional long short-term memory (Bi-LSTM) and long short-term memory (LSTM). Additionally, we utilized word embeddings with a specific focus on BERT for effective language pretraining. Our study explores the applicability of the BERT model in handling multimodal information. The evaluation of our proposed approach was conducted on a data corpus collected from commercial apps like Flipkart. The results indicate that the BERT-LSTM and BERT Bi-LSTM models have significantly improved sentiment analysis scores. Comparative analyses of emoji converted to text, only textual data, and emoji converted to vectors, all these three case studies demonstrated higher accuracies and F1-Scores for our proposed approach.