Multi-domain recommendation (MDR) aims to enhance recommendation accuracy by leveraging knowledge from various source domains to benefit a target domain using a universal model. However, traditional MDR approaches often fail in leveraging the complex users’ preferences spanning multiple domains, especially in the case of users that are active in more than one domain. Hence, a key problem in MDR is how to better leverage the diverse but disparate user/item interaction data available across multiple domains. We argue that the concept of “model soups”, which involves combining multiple models through averaging, presents a promising method for unifying the users’ profiles and item representations from diverse domains, thereby potentially improving the multi-domain recommendation performance. We propose AdapterSoupRec, a novel approach that employs the “model soup” technique to enrich the multi-modal item contents. Our approach involves averaging the weights of Multi-modal Large Language Models (MLLMs) trained across diverse source domains in order to enhance the recommendation performance in the target domain. To enable a universal item multi-modal representation learning for recommendation purposes, we train an MLLM using the items’ images and descriptions. Then, we aggregate the weights of several MLLMs, each trained with various hyper-parameter configurations. This process is akin to blending various soup ingredients. Furthermore, we incorporate a graph neural network as an adapter into AdapterSoupRec to effectively combine the universal item representations obtained from the MLLM and the users’ preferences across multiple domains. We conduct extensive experiments on three e-commerce datasets and show that AdapterSoupRec consistently improves performance in the target domains over nine existing state-of-the-art recommendation approaches. In particular, AdapterSoupRec outperforms the best baseline, BM3, significantly by up to 18.35%. We also show that the model soup technique further benefits modality fusion for item representation learning and exhibits strong out-of-domain performance in a video recommendation scenario.