Object detection and tracking is a vital area of computer vision that has several applications. Despite considerable progress in this field, real-time and platform-dependent difficulties remain unresolved. This includes recording temporal information of the target and background clutter. As of today, numerous deep learning-based detection and tracking algorithms have been reported in the literature, with substantial gains. To fully exploit the promise of present research in this area, object detection, tracking, and associated problems, are explained first in this chapter, followed by a relevant literature review covering the background of this interesting field. This is followed by a description of modern models, benchmark datasets, and performance metrics. Based on these benchmarks, the comparative results of well-known tracking algorithms found in the literature are presented. Finally, this chapter concludes with future research directions in this field.