A rich RGBD images captioning for scene understanding

In recent years, image captioning and segmentation have gained prominence in computer vision, finding applications in various fields, from autonomous driving to content analysis. While several solutions have been devised to enhance user experiences in navigating their environments, there remains a need for applications that provide detailed textual descriptions of scenes. Most existing models primarily focus on specific tasks, limiting their versatility in different scenarios. In this paper, we propose an innovative approach aimed at enhancing the comprehension of surroundings through image captioning. Our research distinctively offers textual descriptions for each segment within an image, including spatial orientations (e.g., left, right, front). This level of granularity ensures that anyone using our system can capture and comprehend every piece of information present in the image. We further extend the applicability of our solution by training and applying our methodology to theatre dataset. Our results demonstrate improved efficiency compared to state-of-the-art methods, coupled with the provision of more detailed descriptions for each segment within the input image.