Multi-branch Residual Fusion Network for Imbalanced Visual Regression

Imbalance visual regression is an important and challenging task, and research on it is still in its early stages. Among the existing solutions, either additional calibration layers need to be added to calibrate the label distribution or feature distribution of the imbalanced data, or a multi-stage training method with a loss function is used to mitigate the imbalance of the data, all of which cause significant time loss. In this paper, we propose an end-to-end multi-branch dynamic residual fusion network (MBDRFN) for imbalance visual regression. MBDRFN consists of a multi-branch residual fusion module (MBRF) and a dynamic balance factor (DBF) that works in concert to overcome the challenge of imbalanced visual regression. The MBRF module has three specialized branches corresponding to three magnitudes of sample labels, and can optimize each branch in a targeted manner to ensure overall fitting performance. The DBF builds on the MBRF and guides the model to gradually focus on learning and optimizing rare sample labels during the training process. Then, we use residual fusion to aggregate the loss functions of dedicated and master branches. The proposed MBDRFN method not only ensures that the model performs well on frequent and regular sample labels, but also improves its performance on rare sample labels with no significant time loss. Finally, we conduct extensive experiments on two real-world datasets to demonstrate the effectiveness and superiority of the proposed MBDRFN method.