This study systematically investigates the challenge of suboptimal semantic segmentation performance in intelligent driving systems under nighttime and low-light conditions. In nighttime scenarios, traditional RGB images suffer from severe degradation in information quality, leading to a sharp decline in model performance, compounded by the scarcity of large-scale annotated nighttime data. Event cameras, with their unique advantages such as high dynamic range and high temporal resolution, offer a novel pathway to address this issue. Consequently, the primary objective of this research is to develop an efficient cross-modal unsupervised domain adaptation (UDA) method for semantic segmentation. By deeply fusing RGB images and event camera data, this method aims to significantly enhance the segmentation accuracy and robustness of models in unlabeled nighttime target domains. To achieve this objective, the study, based on the SegFormer and DAFormer frameworks, proposes two key innovations. First, a cross-modal deep fusion module, AttentionFusion, based on multiple attention mechanisms was designed to facilitate dynamic and adaptive information interaction between image and event features. Second, a pixel-level contrastive learning framework was introduced and optimized to enhance the class discriminability and domain invariance of the learned feature representations. On the benchmark dataset transitioning from Cityscapes (source domain) to DSEC-Night (unlabeled target domain), the proposed method demonstrated significant performance improvements: the mean Intersection over Union (mIoU) increased from 50.28% to 53.44%, and pixel accuracy (aAcc) rose from 83.88% to 85.42%.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Cross-Modal Unsupervised Domain Adaptation Framework for Semantic Segmentation in Low-Light Road Scenes

  • Bowen Sun,
  • Hua Cui,
  • Zhao Yang,
  • Chengkang Duan,
  • Jingyi Gao

摘要

This study systematically investigates the challenge of suboptimal semantic segmentation performance in intelligent driving systems under nighttime and low-light conditions. In nighttime scenarios, traditional RGB images suffer from severe degradation in information quality, leading to a sharp decline in model performance, compounded by the scarcity of large-scale annotated nighttime data. Event cameras, with their unique advantages such as high dynamic range and high temporal resolution, offer a novel pathway to address this issue. Consequently, the primary objective of this research is to develop an efficient cross-modal unsupervised domain adaptation (UDA) method for semantic segmentation. By deeply fusing RGB images and event camera data, this method aims to significantly enhance the segmentation accuracy and robustness of models in unlabeled nighttime target domains. To achieve this objective, the study, based on the SegFormer and DAFormer frameworks, proposes two key innovations. First, a cross-modal deep fusion module, AttentionFusion, based on multiple attention mechanisms was designed to facilitate dynamic and adaptive information interaction between image and event features. Second, a pixel-level contrastive learning framework was introduced and optimized to enhance the class discriminability and domain invariance of the learned feature representations. On the benchmark dataset transitioning from Cityscapes (source domain) to DSEC-Night (unlabeled target domain), the proposed method demonstrated significant performance improvements: the mean Intersection over Union (mIoU) increased from 50.28% to 53.44%, and pixel accuracy (aAcc) rose from 83.88% to 85.42%.