High-mix, low-volume manufacturing environments, particularly in the automotive sector, require precise defect detection and rework processes to maintain quality standards while accommodating workforce variability. Traditional training methods often rely on physical tools and static instructions, which hinder efficiency and limit adaptability across different skill levels. Addressing these challenges, this research introduces a multi-modal guidance system that supports both novice and expert workers by providing real-time, skill-adaptive assistance. The proposed system integrates camera-based projectors and interactive graphical user interfaces (GUIs) to deliver intuitive, dynamic guidance tailored to user expertise. Operating in two distinct modes, Novice Mode enables trainees to mark defects on sub-assembled parts using real-time hand tracking, with projected visual feedback eliminating the need for physical markings. Expert Mode projects defect locations and rework instructions directly onto parts, complemented by GUI-based insights for precise corrections. Laboratory tests across various parts and approximately 10 defect types demonstrate the system’s effectiveness in improving training outcomes and streamlining quality control processes. Key results highlight enhanced efficiency, repeated use of training parts, and improved user engagement through features such as dynamic interaction, 3D model integration, and gamification. The modular and scalable design of the system lays a foundation for intelligent workplace assistance, with future implications for broader adoption in diverse industrial settings to boost productivity and adaptability.