History Based Incremental Singular Value Decomposition for Background Initialization and Foreground Segmentation

Background initialization is an essential step for both hand-crafted and deep learning foreground segmentation approaches. In this paper, we propose a low-rank approximation algorithm that effectively handles the challenge caused by Stationary Foreground Objects (SFOs) on both offline and online bases. The proposed algorithm employs different incremental decomposition mechanisms that control the contribution of earlier and current frames in the overall covariance of the processed video. The proposed algorithm is able to identify the type of the detected SFO, whether it is an abandoned or removed object. Moreover, a background-updating mechanism is introduced to feed the proper background to learning models that are pretrained for foreground segmentation. The experimental results demonstrate the effectiveness of both proposed mechanisms: the SFO identification and the background initialization.