<p>Accurate prediction of postoperative complications in lung cancer patients is crucial for developing clinical treatment plans. Multimodal medical data, such as physiological indicators and medical imaging data, are complementary and can improve prediction accuracy. While traditional methods for utilizing multimodal data have limitations, the application of advanced multimodal large models presents a promising avenue for this task. This paper proposes an integrated framework that employs image embedding models (vision transformer and multi-layer perceptron) to extract feature patches from medical imaging, localizing them for supplementation with text features derived from a Bayesian graph embedding model. This approach aims to achieve unified multimodal features by inputting them into a large language model (LLM) for training. Additionally, it automates the image-text data pipeline by feeding these fused features into the LLM to facilitate multimodal information interaction and fusion. Finally, the proposed solution uses a multi-layer perceptron and a softmax output layer to predict postoperative complications, while retaining the chatbot functionality of the base model to provide medical chat services. The results of ablation tests demonstrate that multimodal fusion improves the accuracy of postoperative complication prediction over unimodal approaches, thereby indicating the potential of the proposed solution.</p>