PIMSeg: Point Cloud Segmentation via 2D Image Mapping and Multimodal Feature Integration
摘要
We propose a multimodal feature fusion network for 3D point cloud semantic segmentation called PIMSeg. Traditional point cloud segmentation methods rely solely on point cloud data, which may lack critical contextual information. By incorporating an additional data modality (RGB images), we utilize complementary information to enhance segmentation performance. To extract global semantic information, PIMSeg projects the original point cloud into a 2D modality containing multi-view images. Within the PIMSeg framework, the original point cloud is processed with corresponding image data. PIMSeg introduces an alignment-fusion module – PIM, which aligns and fuses point cloud features with image features, effectively enhancing the effect of multimodal feature fusion. Additionally, to improve the ability to learn from noisy web data, we employ momentum distillation. We have evaluated the proposed method on the S3DIS and ScanNet datasets.