With the rapid development of smart grids, power operation and maintenance tasks are becoming increasingly complex, involving multi-scene, multi-object, and high-risk scenarios. Traditional rule-based systems lack the adaptability and generalization capacity to handle dynamic inspection environments. To address this, we propose a hierarchical collaborative AI agent architecture tailored for power inspection, integrating a multi-modal large language model (MLLM) with multiple task-specific visual expert modules based on Faster R-CNN. The system establishes a unified workflow of “expert scheduling – perception fusion – strategy generation” to achieve accurate detection of hidden hazards and automatic dispatch strategy formulation. To improve the quality of generated strategies, we design a two-stage training mechanism combining supervised fine-tuning and Direct Preference Optimization (DPO) based on human feedback. Experimental results demonstrate that the proposed system achieves high precision and robustness in detecting insulator damages and transmission channel intrusions. Additionally, the strategy generation module significantly outperforms baseline models in compliance, practicality, and clarity, showcasing strong adaptability and deployment potential in real-world industrial scenarios.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Multi-modal AI Agent for Power Inspection: Adaptive Mapping and Strategy Optimization

  • Duo Huang,
  • Jianyang Sun,
  • Shilin Luo,
  • Youfang Ye,
  • Weicheng Hong,
  • Deming He,
  • Rushan Lin,
  • Zhiyuan Zhuang,
  • Youli Chen,
  • Yifeng Chen

摘要

With the rapid development of smart grids, power operation and maintenance tasks are becoming increasingly complex, involving multi-scene, multi-object, and high-risk scenarios. Traditional rule-based systems lack the adaptability and generalization capacity to handle dynamic inspection environments. To address this, we propose a hierarchical collaborative AI agent architecture tailored for power inspection, integrating a multi-modal large language model (MLLM) with multiple task-specific visual expert modules based on Faster R-CNN. The system establishes a unified workflow of “expert scheduling – perception fusion – strategy generation” to achieve accurate detection of hidden hazards and automatic dispatch strategy formulation. To improve the quality of generated strategies, we design a two-stage training mechanism combining supervised fine-tuning and Direct Preference Optimization (DPO) based on human feedback. Experimental results demonstrate that the proposed system achieves high precision and robustness in detecting insulator damages and transmission channel intrusions. Additionally, the strategy generation module significantly outperforms baseline models in compliance, practicality, and clarity, showcasing strong adaptability and deployment potential in real-world industrial scenarios.