Context-Aware Enhanced Virtual Try-On Network with fabric adaptive registration

Image-based virtual try-on technology provides a better shopping experience for online customers and holds immense commercial value. However, existing methods face challenges in accurately aligning garments and preservation of garment texture details when dealing with challenging body poses or complex target clothing. Furthermore, these methods have been unable to adaptively fuse and generate images based on different body parts in a refined manner, struggling to generate and retain high-quality details of body parts, resulting in limited quality of try-on results. To address these issues, we propose a novel virtual try-on network named Context-Aware Enhanced Virtual Try-On Network (CAE-VTON). The key ideas of our method are as follows: (1) Introducing a Multi-Scale Neighborhood Consensus Warp Module (MNCWM) with matching filtering capability that is sensitive to small semantic differences, which generates highly accurate garment alignment results and coupled natural try-on generation results. (2) Proposing a fabric deformation energy smoothness loss to constrain local deformations of clothing, thus preserving complex details in garments. (3) Designing a Body Reconstruction Module (BRM) that adaptively generates and retains exposed skin areas of the body. (4) Introducing a novel try-on generation module called Context-Adaptive Awareness-Enhanced Try-on Module (CAAETM) that integrates all components and utilizes target semantic label map to adaptively generate the final try-on results for different body parts. We evaluate our model on the VITON-HD and VITON datasets and find that our method achieves state-of-the-art performance in qualitative and quantitative evaluations for virtual try-on.