Communication overhead is a key challenge in distributed deep learning. This paper introduces DSL-SGD, a distributed training scheme with a local update mechanism. DSL-SGD allows local weights to participate in multiple steps until global updates are completed, mitigating communication latency. To reduce parameter differences, it accumulates and averages gradients from multiple steps to update local weights. Experiments on a 32-GPU cluster show DSL-SGD matches the convergence accuracy of synchronous SGD while reducing end-to-end time by 7.9 \(\times \) compared to synchronous SGD and by 71.8% compared to Local-SGD. It also demonstrates superior scalability and computational efficiency by reducing weight updates by a factor of \(k\) .