Scholarlink

<p>Museums and art galleries use modern technologies to engage visitors and attract not only specialists but also the general public, by providing recommendations for interactive studying and observing their collections. These recommendations can be created on the base of artwork similarity defined by using fine-tuned deep neural networks. In this paper, we explore the possibilities of fine-tuning foundation models using the Low-Rank-Adaptation (LoRA) fine-tuning technique for the classification task and perform a similarity search based on features extracted with fine-tuned models. Using LoRA technique allows to use and switch easily between several fine-tuned models with only one frozen backbone, this is useful in the case of using large models on mobile devices with limited storage space. During fine-tuning, we examined the influence of hyperparameters on two DINOv2 models’ performance and found their reasonable combination according to a number of trainable parameters and performance, achieved on the relatively small artistic dataset. We achieved state-of-the-art accuracy for genre classification on the WikiArt dataset with the proposed approach.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Artwork recommendations guided by foundation models: survey and novel approach

  • 期刊论文

  • 发表日期：

    2025年1月2日

查看全文
  • Tetiana Yemelianenko,
  • Iuliia Tkachenko,
  • Tess Masclef,
  • Mihaela Scuturici,
  • Serge Miguet

摘要

Museums and art galleries use modern technologies to engage visitors and attract not only specialists but also the general public, by providing recommendations for interactive studying and observing their collections. These recommendations can be created on the base of artwork similarity defined by using fine-tuned deep neural networks. In this paper, we explore the possibilities of fine-tuning foundation models using the Low-Rank-Adaptation (LoRA) fine-tuning technique for the classification task and perform a similarity search based on features extracted with fine-tuned models. Using LoRA technique allows to use and switch easily between several fine-tuned models with only one frozen backbone, this is useful in the case of using large models on mobile devices with limited storage space. During fine-tuning, we examined the influence of hyperparameters on two DINOv2 models’ performance and found their reasonable combination according to a number of trainable parameters and performance, achieved on the relatively small artistic dataset. We achieved state-of-the-art accuracy for genre classification on the WikiArt dataset with the proposed approach.