Sentiment analysis of memes is crucial in domains such as finance and politics, but the focus has been mainly on English. This study presents our test dataset MemoSen, and a proposed bimodal system dedicated to memes, integrating both text and image, with three sentiment labels: positive, negative, and neutral. A detailed annotation manual is provided to facilitate the development of new resources in this area, thus promoting the extension of sentiment analysis to various languages. Our first model, which integrates a Convolutional Neural Network (CNN), branches to process both images and text using a concatenation merge, and achieves a validation performance of an Area Under the Curve (AUC) about 100%. The second model, based on VGG16, achieved an AUC of 99.56%, while the third model, based on ResNet50, achieved an AUC of 98.69%.