Generation of sentences to the images like human is an important task in computer vision which this task can also be called as image caption. This process bridges the computer vision to natural language processing tasks which generate descriptions to the images. This paper highlights to extract semantical and contextual features from images to capture the effective relationship features with Vision Transformer (ViT) in the form of high-level features like objects, meaningful scenes, attributes from the visionary images. These extracted features particularly from the images helps in deep level understanding to the user, that shows context features and spatial feature information from the images. On these extracted essential features this paper utilizes attention mechanism to particularly focus on dynamic regions that helps the model to generate more semantical and contextual text descriptions to the images. Then these integrated features are utilized by the proposed model and enhances the ability to produce captions that are very fluent and descriptions human-like sentences. A standard dataset has been utilized to experiment and to generate captions such like Flickr8k that resembles the effectiveness in its method in generation of descriptions like humans for improving performance over other baseline methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Generation of Semantic and Contextual Captions to the Images Using ViT

  • Andukuri V. S. S. P. S. S. Srinivas,
  • S. N. TirumalaRao,
  • K. V. Narasimha Reddy,
  • Shaik Rafi,
  • Sathyam Reddy

摘要

Generation of sentences to the images like human is an important task in computer vision which this task can also be called as image caption. This process bridges the computer vision to natural language processing tasks which generate descriptions to the images. This paper highlights to extract semantical and contextual features from images to capture the effective relationship features with Vision Transformer (ViT) in the form of high-level features like objects, meaningful scenes, attributes from the visionary images. These extracted features particularly from the images helps in deep level understanding to the user, that shows context features and spatial feature information from the images. On these extracted essential features this paper utilizes attention mechanism to particularly focus on dynamic regions that helps the model to generate more semantical and contextual text descriptions to the images. Then these integrated features are utilized by the proposed model and enhances the ability to produce captions that are very fluent and descriptions human-like sentences. A standard dataset has been utilized to experiment and to generate captions such like Flickr8k that resembles the effectiveness in its method in generation of descriptions like humans for improving performance over other baseline methods.