Ensuring the structural integrity of containers is essential for safe and reliable logistics. However, existing vision-based defect detection methods often suffer from three major limitations: weak localization of small and subtle defects, vulnerability to background interference, and insufficient inference efficiency for real-world deployment. Motivated by these challenges, this paper proposes CDD-YOLO11, an efficient and robust way based on the YOLO11 architecture. To achieve more robust small-defect localization and efficient model deployment, the proposed approach introduces a Neuron Relevance Attention Module (NRAM) to enhance intra-channel saliency without increasing model complexity. To further improve multi-scale representation and robustness to background noise, we design an Advanced Feature Aggregation Module (AFAM), which integrates a Dynamic Scale Synthesizer (DSS) for adaptive scale-aware fusion, a Multi-scale Feature Encoding (MFE) component for coarse-to-fine spatial encoding, and a Region-Channel Attention Module (RCAM) for precise spatial-channel enhancement. Furthermore, to enhance training stability and localization precision, we use Efficient IoU (EIoU) loss to accelerate convergence and improve bounding box regression accuracy. We also propose a container defect dataset containing 3,828 container door handle catch images. Experiments demonstrate that CDD-YOLO11 increases mAP@0.5:0.95 by 3.0% and recall by 4.7% over the YOLO11-m baseline, confirming its superior accuracy and consistency in industrial scenarios.