As nowadays the targets detection and tracking of ships in complex inland water areas are mainly based on radars and AIS (Automatic Identification System), the accuracy and update rate of automatic targets detection and tracking could not support the auto-pilot navigation. In order to conduct the image object detection and tracking of ships in complex inland water areas, the image object detection based on YOLO v8 network and Byte Track algorithm were researched. The samples augmentation and image object detection network training method were putforward. Based on 2-step Byte track framework, the target tracking model was built. To prove the accuracy and practicality of this method, an experiment was conducted on ‘Changhanghuoyun 001’ vessel which operated on Yangtze River, the result shows that different kinds of targets including ships and beams were detected accurately. Compared with other sensors tracking trajectories, the 2-step Byte track framework could gain the trajectories with the same accuracy. The targets detection and tracking based on image sensors could used on ships in complex inland water areas.