Industrial defect detection is of great significance in enhancing the level of manufacturing industries and achieving the high-end development of manufacturing. Segmentation is an important technology to defect defects. While complex and variable backgrounds, subtle and small defects, class imbalance and sparseness make it difficult to accurately identify defects. The article proposes Anomaly-enhanced Bilateral Segmentation Network (AE-BiseNet) to improve segmentation performance by feature contrasting spatial path with spatial defect enhancement auxiliary loss and local positive and negative representation contrasting auxiliary loss. The former creates a dual-branch structure to capture rich low-level spatial features and trained with auxiliary loss to distinguish normal area and defects. The latter extract the representative high-level features of normal area and defects and mitigate class imbalance problem. Both the auxiliary loss function use ground truth to help confirm normal features and defect features. The stamped parts data set is built to validate the model. Experiments show the proposed method reach mean IoU of 83.8% and defect IoU of 68.0% on the stamped parts data set and mean IoU of 88.1% and defect IoU of 79.4%.