<p>Multi-focus image fusion is a crucial technique for generating a composite image in which all objects appear in focus. The core challenge of this task lies in designing effective activity-level metric strategies and decision map refinement methods. This paper proposes a robust technique, the selective scale invariant feature transform (SSIFT), to address both issues. Specifically, we innovatively introduce a directional selection mechanism on the activity level map, retaining only the K directional components with the strongest response intensity within each local sub-block. This reduces directional feature redundancy and supports robust initial segmentation. Furthermore, for decision refinement, we combine image segmentation algorithms to refine the decision map, thereby achieving more robust image fusion. Comparative experiments on public datasets demonstrate that the proposed method performs excellently in both qualitative and quantitative analyses.</p>