Sentiment analysis has applications in many fields where there is a need for emotion recognition without human intervention. The advances in deep learning implementation have led to higher-performing classification systems for complex input data. From all data modalities that can be used for emotion classification, this study focuses on the use of visual data for sentiment analysis. In this paper, we propose a hybrid system by combining a pre-trained CNN for visual feature extraction and a recurrent neural network for encoding temporal data. The CREMA-D dataset was used to train machine learning models and to validate the system. The results of the CREMA-D visual dataset approach are better than other works.