A Multi-modal AI Agent for Power Inspection: Adaptive Mapping and Strategy Optimization
摘要
With the rapid development of smart grids, power operation and maintenance tasks are becoming increasingly complex, involving multi-scene, multi-object, and high-risk scenarios. Traditional rule-based systems lack the adaptability and generalization capacity to handle dynamic inspection environments. To address this, we propose a hierarchical collaborative AI agent architecture tailored for power inspection, integrating a multi-modal large language model (MLLM) with multiple task-specific visual expert modules based on Faster R-CNN. The system establishes a unified workflow of “expert scheduling – perception fusion – strategy generation” to achieve accurate detection of hidden hazards and automatic dispatch strategy formulation. To improve the quality of generated strategies, we design a two-stage training mechanism combining supervised fine-tuning and Direct Preference Optimization (DPO) based on human feedback. Experimental results demonstrate that the proposed system achieves high precision and robustness in detecting insulator damages and transmission channel intrusions. Additionally, the strategy generation module significantly outperforms baseline models in compliance, practicality, and clarity, showcasing strong adaptability and deployment potential in real-world industrial scenarios.