Even though Multi-modal Large Language Models (MLLMs) achieve success in our daily lives, they show vulnerability to jailbreak attacks. MLLMs need to handle various modalities of data such as language and vision, offering new perspectives for jailbreak attacks. This paper proposes a multi-modal collaborative jailbreak attack method. Since MLLMs encode input image into embeddings aligned with text, these embeddings can be considered to contain a certain amount of semantic information. Our goal is to perturb the input image so that the semantic information in these embeddings includes content related to the jailbreak objective, thereby assisting the language modality in executing jailbreak attacks. Specifically, in order to collaborate with the input jailbreak instruction, we encode harmful content related to the jailbreak instruction into the input image, thereby increasing the probability of the model generating harmful content. Additionally, we utilize cross-attention to identify critical regions in the image that are more relevant to the jailbreak instruction. Then correspondingly amplify the magnitude of perturbations, ensuring that the harmful content contained in the image is more fully represented. Extensive experiments on various MLLMs, including MiniGPT-4, InstructBLIP and LLaVA, strongly support that our method outperforms the comparisons by large margins.