Optical Flow Generation Method Based on Multi-scale Feature Enhancement of Compressed Video Bitstream Information
摘要
Most existing deep learning based optical flow estimation methods use uncompressed video frames to calculate optical flow, while existing application videos are compressed, and the compressed video stream information contains motion vectors (MV). Both MV and optical flow represent the motion information of two frames, but MV is a dispersed field, while optical flow is a dense field. The existing methods directly use MV to estimate optical flow, which can cause a decrease in accuracy, The residual (RES) in compressed information can enhance the conversion performance from discrete MV to dense optical flow. In addition, due to the inability of single scale convolution processes to effectively associate deep semantic features of different data, thus this paper proposes a multi-scale fine-grained generation network that uses RES compensated MV for optical flow generation, solving the problem of low accuracy in existing methods. This scheme has achieved good performance through endpoint error benchmark testing in the Sintel dataset.