Hidden danger identification and analysis algorithm combining multi-modal large model and knowledge enhancement
摘要
The criteria for evaluating the risks of significant accidents are crucial for directing enterprises to assume their primary responsibility for production safety, perform self-assessments and rectify potential safety hazards, and for regulatory and law enforcement agencies to execute precise and effective enforcement of laws. This work presents an approach for hazard detection and analysis that integrates multimodal big models with knowledge augmentation. This algorithm initially augments the training samples by leveraging the comprehensive knowledge of the multimodal large model, creates an instruction fine-tuning dataset for metaphor interpretation, and performs contrastive embedding training utilizing unlabeled data to improve the semantic representation capacity of the text encoder. The multimodal big model is subsequently fine-tuned using varied instructional data, including knowledge development, possible hazard material, and metaphor recognition, to boost its capacity for recognizing potential dangers. Ultimately, self-predictive iterative expansion training is performed on unlabeled data using an accelerated learning process to improve the model's flexibility to certain tasks. The experimental findings indicate that, under identical resource constraints, the suggested model outperforms the baseline technique on the accessible datasets. Particularly when the quantity of labeled data is constrained, the accuracy rate has risen by around 21%. The ablation tests confirmed the substantial impact of the knowledge improvement, self-iterative training, and dynamic gating fusion modules on model performance. This research presents an algorithm that offers an interpretable and resilient approach for the intelligent detection of possible risks, therefore considerably improving metaphor parsing and adaptation in low-resource contexts.