Counting of vehicles has interesting and important applications in many areas including visual surveillance, urban landscaping for effective utilization of limited space in cities for parking, traffic management and assisting investigation teams to find stolen vehicles. Existing models reported are extensively focused on type of vehicles and scenes, whereas our work focuses on vehicle counting in complex scenarios involving multiple types of vehicles with clutter backgrounds. The proposed Special Residual Global Attention Module (SRGAM) is integrated with the Detection Transformer (DETR) for accurate vehicle counting in different situations, given the high success rate of transformers in object detection. The proposed work, to extract invariant features, integrates the ResNet50 with SRGAM for multiple types of vehicle counting. The efficacy of the proposed method is demonstrated by testing on our dataset, IIITDWD and benchmark dataset, MSCOCO2017. The results obtained on the dataset show that the proposed method outperforms the state-of-the-art methods in terms of mean precision.