The recently developed DEtection TRansformer (DETR) establishes a new paradigm of object detection with a set of learnable queries. However, this paradigm still lags in performance behind CNN-based detectors. The limited performance is largely attributed to redundant predictions among queries and the supervision sparsity on predictions. In this paper, we present a novel MCPA-DETR with the corresponding two contributions. Firstly, we propose a query-aware region constraint to make the responsibility region of each query more focused. Secondly, we introduce a progressive label assignment to provide richer supervision signals, promoting the convergence speed. Our MCPA-DETR is efficient due to without introducing extra parameters and computational overhead. Extensive experiments validate the effectiveness of the proposed method, and our method can generalize well across existing DETR-based models. On the challenging MS COCO benchmark, our MCPA-DETR outperforms the existing models by a large margin.