EDOM-MFIF: an end-to-end decision optimization model for multi-focus image fusion

In the multi-focus image fusion process, generating a focus decision map is typically a requisite step before the actual fusion of images. Many multi-focus image fusion methods often require optimization through post-processing techniques such as hole filling and removal of small regions in the decision map. To solve these problems and obtain better fusion images, we propose a multi-focus image fusion algorithm based on an end-to-end decision optimization model called EDOM-MFIF. Firstly, we design a cross-scale feature extraction encoder based on the joint attention fusion of upper and lower information, which can fully extract the shallow texture information and high-level semantic information of images. Secondly, we design a feature fusion module using convolution and spatial attention for comprehensive feature fusion and maximal retention of global image information. Finally, we achieve end-to-end training and optimization of the focusing decision map by integrating the decision map’s optimization into the whole image fusion architecture. This prevents evident pixel faults in the immediate area from spreading to other areas during post-processing, which could lead to inaccurate classification findings, in addition to maintaining pixel consistency with the original image to the greatest extent possible. To facilitate training of the multi-focus image fusion model, we construct a large-scale multi-focus image dataset with a supervised decision map and test the algorithm on three different types of public datasets. The experimental results show that the proposed algorithm outperforms other advanced multi-focus image fusion algorithms in both objective evaluation and visual evaluation.