Scholarlink

<p>The development of accurate vessel detection systems for Unmanned Aerial Vehicles (UAVs) is severely constrained by the scarcity and limited diversity of annotated training data, particularly for rare vessel types and complex maritime scenarios. This study introduces a generative data augmentation framework to address these limitations through two complementary image synthesis methods. The first method employs a multimodal language model to produce diversified scene-level descriptions, which are then used as prompts for a diffusion model to synthesize realistic vessel-centric images under varying environmental and contextual conditions. The second method targets rare object classes by segmenting real instances using SAM with a context-aware co-segmentation strategy and compositing them onto diverse backgrounds generated by the diffusion model, guided by language-based prompts. All synthetic images are annotated using a hybrid zero-shot strategy that fuses YOLO-World and Grounding DINO predictions through a confidence-weighted mechanism. Experiments on the VesselImg benchmark dataset demonstrate notable performance gains in vessel detection tasks, especially when using lightweight models suitable for UAV deployment. The YOLO11 detector achieved a mean average precision (mAP) improvement from 0.785 to 0.828 at IoU thresholds of 0.50:0.95, while the rare “Pilot” class showed an increase in average precision from 0.534 to 0.617. These results validate the effectiveness of the proposed approach in enhancing vessel detection robustness and highlight its potential as a scalable solution for mitigating data limitations in maritime AI applications. The code and dataset of this study are publicly available on GitHub at: <a href="https://github.com/lethithuhong1302/UAVVesselDiffusion">https://github.com/lethithuhong1302/UAVVesselDiffusion</a></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Addressing data scarcity and imbalance in vessel detection from UAV-captured images via diffusion-based multimodal synthesis and context-aware compositing

  • 期刊论文

  • 发表日期：

    2026年1月28日

查看全文
  • Chi-Thanh Nguyen,
  • Thu-Huong Pham,
  • Thi-Thu-Hong Le

摘要

The development of accurate vessel detection systems for Unmanned Aerial Vehicles (UAVs) is severely constrained by the scarcity and limited diversity of annotated training data, particularly for rare vessel types and complex maritime scenarios. This study introduces a generative data augmentation framework to address these limitations through two complementary image synthesis methods. The first method employs a multimodal language model to produce diversified scene-level descriptions, which are then used as prompts for a diffusion model to synthesize realistic vessel-centric images under varying environmental and contextual conditions. The second method targets rare object classes by segmenting real instances using SAM with a context-aware co-segmentation strategy and compositing them onto diverse backgrounds generated by the diffusion model, guided by language-based prompts. All synthetic images are annotated using a hybrid zero-shot strategy that fuses YOLO-World and Grounding DINO predictions through a confidence-weighted mechanism. Experiments on the VesselImg benchmark dataset demonstrate notable performance gains in vessel detection tasks, especially when using lightweight models suitable for UAV deployment. The YOLO11 detector achieved a mean average precision (mAP) improvement from 0.785 to 0.828 at IoU thresholds of 0.50:0.95, while the rare “Pilot” class showed an increase in average precision from 0.534 to 0.617. These results validate the effectiveness of the proposed approach in enhancing vessel detection robustness and highlight its potential as a scalable solution for mitigating data limitations in maritime AI applications. The code and dataset of this study are publicly available on GitHub at: https://github.com/lethithuhong1302/UAVVesselDiffusion