<p>Multimodal fusion under smoke conditions faces the challenge of being unable to obtain clear fused images. Traditional fusion methods assume clear imaging conditions and fail to address nonlinear degradation. Existing stepwise smoke removal fusion processes are complex, prone to error accumulation, and lack lightweight solutions that balance performance and efficiency. This paper proposes a direct smoke removal multimodal image fusion framework based on two stage training and a lightweight CNN-Transformer architecture. The method employs an encoder-decoder backbone. The encoder combines CNN for local feature extraction and Transformer for global dependency modeling. A lightweight latent feature mapping network has achieved smoke suppression. Systematic ablation experiments validate the network structure and fusion strategy. Experiments on a multimodal test dataset with varying smoke concentrations show that the proposed method outperforms current mainstream fusion methods in PSNR, SSIM, MSE, and ASC. This research provides a new technical pathway for lightweight and efficient of multimodal perception in complex harsh environments.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A lightweight multimodal image fusion and enhancement method for smoke scenes

  • Jie Ma,
  • Ming Yang,
  • Xianli Jin,
  • Yangyang Zhao

摘要

Multimodal fusion under smoke conditions faces the challenge of being unable to obtain clear fused images. Traditional fusion methods assume clear imaging conditions and fail to address nonlinear degradation. Existing stepwise smoke removal fusion processes are complex, prone to error accumulation, and lack lightweight solutions that balance performance and efficiency. This paper proposes a direct smoke removal multimodal image fusion framework based on two stage training and a lightweight CNN-Transformer architecture. The method employs an encoder-decoder backbone. The encoder combines CNN for local feature extraction and Transformer for global dependency modeling. A lightweight latent feature mapping network has achieved smoke suppression. Systematic ablation experiments validate the network structure and fusion strategy. Experiments on a multimodal test dataset with varying smoke concentrations show that the proposed method outperforms current mainstream fusion methods in PSNR, SSIM, MSE, and ASC. This research provides a new technical pathway for lightweight and efficient of multimodal perception in complex harsh environments.