Generation of Semantic and Contextual Captions to the Images Using ViT
摘要
Generation of sentences to the images like human is an important task in computer vision which this task can also be called as image caption. This process bridges the computer vision to natural language processing tasks which generate descriptions to the images. This paper highlights to extract semantical and contextual features from images to capture the effective relationship features with Vision Transformer (ViT) in the form of high-level features like objects, meaningful scenes, attributes from the visionary images. These extracted features particularly from the images helps in deep level understanding to the user, that shows context features and spatial feature information from the images. On these extracted essential features this paper utilizes attention mechanism to particularly focus on dynamic regions that helps the model to generate more semantical and contextual text descriptions to the images. Then these integrated features are utilized by the proposed model and enhances the ability to produce captions that are very fluent and descriptions human-like sentences. A standard dataset has been utilized to experiment and to generate captions such like Flickr8k that resembles the effectiveness in its method in generation of descriptions like humans for improving performance over other baseline methods.