The textile industry faces significant challenges in quality control due to the complexity of detecting subtle defects in fabrics during high-speed and high-volume production. Traditional inspection methods rely heavily on human inspection, which is labor-intensive, error-prone, and unable to keep pace with modern manufacturing speeds. This paper presents an advanced Computer Vision and Deep Learning-based anomaly detection system for textile quality assurance that addresses these limitations. By leveraging ensemble Convolutional Neural Networks (CNNs), Autoencoders, and Vision Trans-formers, integrated with IoT-enabled sensors and edge computing devices, our model achieves superior defect detection accuracy across diverse fabric types (weft and warp defects, slubs, holes, oil stains, mispicks, color incon-sistencies, broken ends, and fabric wrinkles etc.) and environmental conditions. The system improves supply chain efficiency by 27%, reduces material waste by 32%, and ensures consistent quality with minimal human intervention. Experimental results demonstrate the effectiveness of CNN variants such as ResNet, EfficientNet-B3, and Vision Transformers, achieving an accuracy of over 97.13% with a false positive rate of less than 2.8%. The proposed framework is scalable, energy-efficient, and deployable in real-world textile manufacturing units with minimal disruption to existing workflows. Cost–benefit analysis indicates a return on investment within 8–12 months of implementation.