Distributed Multimodal Video Generation and Interaction Framework Based on Message-Driven Middleware
摘要
Emerging applications such as virtual customer service, online education, and digital humans demand real-time generation of multimodal content with stringent requirements on throughput, latency, and reliability. Traditional solutions built on synchronous REST APIs or basic queue models exhibit tight coupling, limited scalability, and weak fault isolation, making them unsuitable for industrial-scale workloads. This paper presents a message-driven distributed framework for multimodal video generation and interaction that decouples task and result flows, supports a multi-producer/multi-consumer pattern, and achieves linear scalability and fault tolerance through asynchronous task orchestration. The pipeline integrates five AI modules in succession—large-language-model dialogue generation, text-to-speech synthesis, Wav2Lip lip synchronization, lightweight semantic motion matching, and MimicMotion-based motion refinement—which together yield lifelike video, even for low-resource languages. Four core roles constitute the system: task producers implemented in Go, which embed trace IDs and push JSON jobs to a video tasks queue; a message broker that handles routing, retries, and dead-letter queues; Python workers that execute the multimodal chain and publish finished videos to a video results exchange; and result listeners that fetch videos by routing key and return them to clients. We detail the architecture, module dependencies, and containerized deployment scripts, and we release complete open-source code with one-click reproduction tools, offering the research and industrial communities a highly reproducible, maintainable, and easily extensible engineering paradigm for multimodal content generation.