Image Caption Generator with CNN and LSTM: A Focus on Descriptive Image Analysis

The goal of this research paper is to collect textual descriptions from images using a set of pre-trained VGG16 models, image processing techniques, and transformer-based models. Encoding the output sequences, mapping image IDs to recipe labels, and a data generator function are all part of the training data preparation process. The encoder and decoder components of the model architecture use cross entropy loss and Adam optimizer. We investigated two training ratios, 80%, and 90%, and trained each for 20 epochs. Evaluation scales, such as BLEU-1 and BLEU-2 scores, are used. The results indicate superior performance with a training rate of 90%. Furthermore, a pipeline that displays precise descriptions of unseen data is implemented by converting images to text using a transformer library. The main functions are described in the Preprocessing section: ‘preprocess_input()} for VGG16 model compatibility, ‘clean()} for cleaning captions, `Tokenizer()} for encoding captions, and ‘pad_sequences} for standardized sequence lengths. Collectively, these functions improve the quality and standardization of input data. In conclusion, there is potential to improve image description synthesis by combining advanced models, especially transformers, with image preprocessing. The paper demonstrates the importance of appropriate preprocessing in enhancing model performance and lays the foundation for future studies in various fields.