The data-parallel distributed training technique is a de facto approach in training large-scale deep neural networks (DNNs) using synchronous stochastic gradient descent (S-SGD). However, S-SGD requires iteratively aggregating the distributed gradients through an AllReduce collective, which easily results in significant data communication across distributed GPUs and thus limits the scaling efficiency of the training system. In this paper, we propose an efficient and practical gradient sparsification and quantization algorithm, named SQ-DeAR, which not only significantly reduces the communication traffic through gradient sparsification and quantization, but also allows overlapping communications with both feed-forward and backpropagation computations through decoupling the communication collective operation. In addition, to improve the computation efficiency of gradient sparsification, we design a batched gradient sparsification to reduce the number of GPU launches. Performance evaluation on a 32-GPU cluster shows that SQ-DeAR outperforms state-of-the-art solutions by 1.17 \(\times \) \( -\) 7.0 \(\times \) .