Scholarlink

<p>Existing Virtual Try-Off (VTOFF) methods struggle with inaccurate reconstruction of garment structures and details. Moreover, high-fidelity generation methods usually require auxiliary mask inputs, which complicate the inference pipeline. To tackle these limitations, we present MMTryOff, a dedicated multi-category unified modeling framework for the VTOFF task. Leveraging the Diffusion Transformer (DiT) as our backbone, our model fuses reference portrait features and category-level text semantic constraints, enabling practical mask-free inference and efficient cross-category garment reconstruction with basic attribute editing capability. To further strengthen structural coherence and fine-grained visual fidelity, we propose a Structure-Detail Co-Optimal approach to collaboratively enhance garment structural integrity and textual details. Concretely, frequency-domain regularization is incorporated to enhance texture authenticity, while automatic spatial mask supervision is deployed throughout training to refine garment structure reconstruction. Furthermore, we construct a large-scale, high-quality dataset, termed VITOFF-HD, through task-specific data filtering, precise alignment, and adaptive augmentation. It contains 148,695 aligned image pairs covering 11 clothing categories, offering task-customized data to facilitate robust model training. Beyond that, VITOFF-HD supports both VTON and VTOFF tasks, and serves as a unified evaluation benchmark for comprehensive virtual clothing manipulation research. Extensive evaluations on VITON-HD, DressCode, and our newly constructed VITOFF-HD demonstrate that our method achieves favorable and competitive performance. The proposed approach can effectively alleviate common visual defects, including structural distortion and blurred fine-grained details. It also exhibits promising robustness under challenging conditions, such as garment occlusion, lace textures, translucent fabrics, openwork structures, and gradient-color garments. We hope that our method and constructed dataset can offer solid baselines and valuable insights for future academic research and practical industrial applications.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

MMTryOff: multi-category virtual try-off with mask-free inference via diffusion transformer

  • 期刊论文

  • 发表日期：

    2026年7月9日

查看全文
  • Mingyue Su,
  • Leilei Li

摘要

Existing Virtual Try-Off (VTOFF) methods struggle with inaccurate reconstruction of garment structures and details. Moreover, high-fidelity generation methods usually require auxiliary mask inputs, which complicate the inference pipeline. To tackle these limitations, we present MMTryOff, a dedicated multi-category unified modeling framework for the VTOFF task. Leveraging the Diffusion Transformer (DiT) as our backbone, our model fuses reference portrait features and category-level text semantic constraints, enabling practical mask-free inference and efficient cross-category garment reconstruction with basic attribute editing capability. To further strengthen structural coherence and fine-grained visual fidelity, we propose a Structure-Detail Co-Optimal approach to collaboratively enhance garment structural integrity and textual details. Concretely, frequency-domain regularization is incorporated to enhance texture authenticity, while automatic spatial mask supervision is deployed throughout training to refine garment structure reconstruction. Furthermore, we construct a large-scale, high-quality dataset, termed VITOFF-HD, through task-specific data filtering, precise alignment, and adaptive augmentation. It contains 148,695 aligned image pairs covering 11 clothing categories, offering task-customized data to facilitate robust model training. Beyond that, VITOFF-HD supports both VTON and VTOFF tasks, and serves as a unified evaluation benchmark for comprehensive virtual clothing manipulation research. Extensive evaluations on VITON-HD, DressCode, and our newly constructed VITOFF-HD demonstrate that our method achieves favorable and competitive performance. The proposed approach can effectively alleviate common visual defects, including structural distortion and blurred fine-grained details. It also exhibits promising robustness under challenging conditions, such as garment occlusion, lace textures, translucent fabrics, openwork structures, and gradient-color garments. We hope that our method and constructed dataset can offer solid baselines and valuable insights for future academic research and practical industrial applications.