MGNet: Mask Guided Transparent Object Depth Completion with Hybrid CNN–Transformer Network
摘要
Accurate depth perception of transparent objects is crucial for robot manipulation. Due to the lack of texture and color, as well as complex optical effects, depth sensors readily produce incomplete depth measurements while observing transparent objects. To obtain their complete depth, one-stage methods that directly regress depth are used, but they are prone to produce blurry and distorted boundaries, limiting the quality of depth completion. In contrast, two-stage methods rely heavily on explicitly predicted masks, but their performance degrades significantly as masks are inaccurate. To mitigate the dependency on explicit masks, we propose here a unified single-stage framework that simultaneously performs transparent object segmentation and depth completion. It employs a hybrid CNN-Transformer backbone designed to jointly extract geometric structures and semantic representations from RGB-D data. Within the decoder, implicit semantic features and explicit mask priors are adaptively used to guide the depth reconstruction process, thereby enhancing boundary precision and overall completion accuracy. Extensive experiments conducted on two public datasets demonstrate the superior effectiveness of the proposed method.