Implementation of a Deep Learning Ensemble Framework Based on Bidirectional Message Queues
摘要
The integration of deep learning algorithms into the main program module is a necessary step for its widespread application in practical production. Combining the data communication characteristics of deep learning modules and the requirements of process communication mechanisms, this paper adopts a process communication mechanism based on bidirectional message queues to integrate the deep learning module with the main program module. By defining bidirectional message queues, message content and structure, lightweight data such as descriptive parameters and control commands are transmitted between modules. Based on the message queue communication mechanism, it overcomes the shortcomings of process communication mechanisms such as high overhead of semaphore mechanisms and relatively complex shared memory processes. It not only meets the requirements of loosely coupled communication between deep learning module and main program module, but also simplifies the communication process. At the same time, the access of massive data such as seismic data and labeled data is centralized within the deep learning module, achieving high cohesion and low coupling. Finally, the coupling degree of the integrated module is analyzed. Taking river intelligent detection as an example, it has been applied in practical production to verify the effectiveness of the deep learning module integration method.