Pixel-Level Segmentation for Multiobject Tracking Using Mask RCNN-FPN

In artificial intelligence (AI), segmenting and tracking multiple objects is a crucial task that has numerous applications in industries such as robots, autonomous vehicles, medical services, and surveillance. Object segmentation identifies and categorizes objects in an image or video frame, while object tracking detects and follows objects throughout a series of frames. These tasks are challenging due to occlusions, varying appearances, and overlapping bounding boxes. This study proposes a novel approach that reduces computing complexity and enhances performance by treating object parts as points. The technique consists of two stages, namely object segmentation and object tracking. In the first stage, a Mask R-CNN model is employed to segment each object in the input image or video frame. The model generates a series of binary masks representing the pixels associated with each item. In the second stage, the segmented objects are treated as points, and a tracking algorithm is applied to monitor each item over time. Based on spatial distance and visual similarity, the Hungarian algorithm is used to compare each object in the current frame to the corresponding object in the previous frame. The proposed method is evaluated using the MOTChallenge benchmark dataset, which includes challenging scenarios such as occlusions and congested scenes. The Simultaneous Object Tracking and Segmentation (SOTS) technique is a highly effective approach for tracking and segmenting objects simultaneously. It employs the Mask RCNN FPN Model and a Point-Track network for object tracking, achieving an impressive F-measure score of 68.8 on the KITTI MOTs dataset,75.9 on the APOLLO MOTS dataset, and 62.3 on the IDD dataset. The method obtains an average Multiple Object Tracking Accuracy (MOTA) of 60.2% and an average segmentation Intersection over Union (IoU) of 70.9%. These results demonstrate that the proposed strategy surpasses current state-of-the-art approaches in terms of accuracy and computational efficiency, with real-time processing speeds of up to 27 frames per second on a single CPU. Therefore, the suggested method holds significant potential for real-world applications in robotics, surveillance, and autonomous vehicles.