Directional Gradient Attacks for Inducing Controllable Detection Errors in DETR-Based Models
摘要
End-to-end object detection models such as DETR and its variants have received widespread attention due to their unique architectural design and impressive detection performance. However, adversarial attacks targeting these models remain relatively underexplored. Existing approaches have primarily focused on traditional two-stage or anchor-based detectors, making them difficult to directly apply to the DETR architecture. To obtain a deeper insight into the behavioral characteristics of DETR-like models under adversarial perturbations, this paper introduces a structured attack strategy based on directional gradient projection, focusing on four categories of attack objectives: object vanishing, misclassification, object fabrication, and random output. This method is implemented through guided projection of gradients toward targeted directions, achieved by intervening in the bipartite matching mechanism, perturbing the decision boundaries of the classification head, reinforcing the self-enhancement mechanism, and disrupting the collaboration of feature streams, thereby enabling the model to generate controllable adversarial misdetections at the output stage. Comprehensive evaluations conducted on the MS COCO dataset reveal that the proposed attack methods cause an average AP reduction of 0.378 across different DETR variants, with the decline reaching 0.408 under the object-vanishing objective. These results significantly degrade detection performance and expose the structural vulnerability of DETR-based models when subjected to adversarial perturbations.