Multi-modal data processing is about exploring the interactions between various types of data to produce a more comprehensive or accurate understanding of a phenomenon such as health, emotions, or circumstances. Vectors as data representation methods have emerged as an important component in modern data management, driven by the growing importance for the need to computationally describe multi-modal data such as texts, images and video in various domains. In this tutorial, we provide a fundamental introduction on vector representations of multi-modal data, which includes intra-modal representation and inter-modal representation. The goal of our tutorial is to provide a centralized and condensed introduction regarding theories and applications of multi-modal data vectorization technologies for both database researchers and practitioners. We also discuss how to use vector database management systems for the management of multi-modal data.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Vector Representations of Multi-modal Data

  • Toni Taipalus,
  • Jiaheng Lu

摘要

Multi-modal data processing is about exploring the interactions between various types of data to produce a more comprehensive or accurate understanding of a phenomenon such as health, emotions, or circumstances. Vectors as data representation methods have emerged as an important component in modern data management, driven by the growing importance for the need to computationally describe multi-modal data such as texts, images and video in various domains. In this tutorial, we provide a fundamental introduction on vector representations of multi-modal data, which includes intra-modal representation and inter-modal representation. The goal of our tutorial is to provide a centralized and condensed introduction regarding theories and applications of multi-modal data vectorization technologies for both database researchers and practitioners. We also discuss how to use vector database management systems for the management of multi-modal data.