Image Caption Generation Using Neural Networks

Image captioning is a growing field and has many applications. This paper focuses on the use of two different models—one for extracting information from images and another for generating captions. They were combined to obtain the final model. The algorithm uses a frequent vocabulary where words whose presence in the dataset is more than a threshold value are kept. These words were indexed for use in the model. All images were passed through the convolution neural network base of the model for extracting images; the model dealing with text generation contains LSTM, a type of recurrent neural network. Embedding vectors from GloVe word embeddings were incorporated for better results. The categorical cross-entropy loss function was used and the values obtained were closely monitored while training the model and the model was tuned accordingly. A METEOR performance metric was used to quantify the performance of the model. The result obtained is comparable to existing models.