Retrieving images with missing regions by fusion of content and semantic features

Deep neural networks with a significant ability to learn and extract image discriminative features make a significant contribution to image retrieval systems. Poor performance in retrieving query images with missing regions is the weak point of image retrieval systems. In this paper, a generative adversarial network is proposed with the aim of inpainting the incomplete images with missing regions in the image retrieval system. Query image inpainting is performed simultaneously at both general and partial levels with two generative networks. Inpainted areas include the semantic and visual features of the input query image. The inpainted image can then be used in the image retrieving system. In the image retrieval process, the content features of the image are extracted from handcrafted features and the VGG-16 deep neural network, including color, texture, and semantic features. The attribute vector of each image is obtained by fusion of the attributes of both parts. Finally, similar images are retrieved based on the smallest Euclidean distance. The explainability of important features of the image in the form of effective super pixels of the image has also been interpreted before and after the use of the LIME technique. The performance of the image retrieval model is confirmed on the ImageNet dataset.