<p>Anomaly detection is demanding for industrial quality assurance and cost reduction, which is still a challenging problem in practice, due to limited availability of anomalies for model training. This work presents a new self-supervised method based on optimal discrete codebook to address defect (anomaly) segmentation on textile fabric. To suppress the generalization of reconstructing anomalous images, an optimal discrete codebook is learned to encode the abnormal feature into a normal one through latent discrete quantization. Moreover, to improve the discriminative power for subtle abnormal regions, the self-supervised segmentation model is presented by combining multi-layer abnormal features instead of reconstruction error. By using the solid color fabric dataset and the denim dataset for validation, the image-level AUROC reaches 100% and 98.8%, respectively; the pixel-level AUROC reaches 96.7% and 94.1%, respectively. The experimental results demonstrate that the proposed method outperforms state-of-the-art self-supervised algorithms in terms of both running time and localization accuracy for fabric defects, which makes it especially suitable for fabric defect segmentation tasks.</p>