错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

AutoPrep-MM: Reinforcement Learning for Multimodal Data Preprocessing with Graph-Based Interaction Modeling

  • Jing Chang,
  • Chang Liu,
  • Jinbin Huang

摘要

Multimodal data preprocessing, cleaning and transforming text, images, and tabular records, which is critical for model performance but remains largely manual. Inappropriate strategies can degrade accuracy by 10–30%, yet optimal pipeline design is prohibitively difficult due to vast search spaces and cross-modal dependencies. Existing AutoML tools treat modalities in isolation or rely on heuristics that ignore the sequential nature of data preparation. We propose AutoPrep-MM, a reinforcement learning framework that automatically optimizes preprocessing pipelines for multimodal data. Our approach employs a Deep Q-Network agent with a Graph Neural Network for cross-modal interaction modeling, which learns to discover latent dependencies between modalities. This enables sophisticated policies that balance downstream task accuracy, data quality, and computational cost. Evaluated on four diverse benchmarks, AutoPrep-MM consistently outperforms state-of-the-art baselines, improving 0.46%–4.10% accuracy while providing interpretable visualizations of learned cross-modal dependencies.