The paper describes the developed prototype system for automated verification of product images for compliance with market requirements. The authors use neural network models as a knowledge model in the prototype. The influence of visual parameters such as background color, margins, sharpness, and absence of artifacts is investigated with respect to compliance assessment as a computer vision task. It is found that traditional object detection methods are insufficient for complete image compliance verification. The authors study the behavior of multimodal models such as CLIP, BLIP-2, and PaliGemma in the context of visual and textual alignment. The authors find that PaliGemma provides acceptable accuracy for image compliance tasks from zero frame. A solution based on product image alignment with formalized text rules is formulated. A comparison between pipeline, full-frame, and end-to-end multimodal approaches is proposed. A software implementation is developed and tested in Roboflow, Google Colab, and local environments. The paper presents an experimental comparison of these approaches in terms of accuracy, resource requirements, and suitability for deployment.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Detection and Verification of the Object's Compliance with the Requirements of Marketplace Using Neural Networks

  • Galina B. Barskaya,
  • Tatiana Y. Chernysheva,
  • Stanislav O. Sbrodov,
  • Alexander O. Tretyak

摘要

The paper describes the developed prototype system for automated verification of product images for compliance with market requirements. The authors use neural network models as a knowledge model in the prototype. The influence of visual parameters such as background color, margins, sharpness, and absence of artifacts is investigated with respect to compliance assessment as a computer vision task. It is found that traditional object detection methods are insufficient for complete image compliance verification. The authors study the behavior of multimodal models such as CLIP, BLIP-2, and PaliGemma in the context of visual and textual alignment. The authors find that PaliGemma provides acceptable accuracy for image compliance tasks from zero frame. A solution based on product image alignment with formalized text rules is formulated. A comparison between pipeline, full-frame, and end-to-end multimodal approaches is proposed. A software implementation is developed and tested in Roboflow, Google Colab, and local environments. The paper presents an experimental comparison of these approaches in terms of accuracy, resource requirements, and suitability for deployment.