The rapid development of large language models across multiple domains have demanded better methods for effective model merging strategies. In order to combine small multimodal models and improve their performance with a reduction in computational cost, the proposed method suggests an evolutionary algorithm-based model merging framework. The two stages of the framework involve layer merging via Passthrough mechanism and layer selection using the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). CMA-ES optimizes the choice of model components by meticulously selecting the most appropriate layers from pre-trained models. Passthrough procedure maintains the advantages from each founding model through mount of specific layers without retraining. The integrated models perform better than the individual models in terms of perplexity, precision, recall, and F1-score, according to experimental results on benchmark datasets including Massive Multitask Language Understanding (MMLU) and MS-COCO. This work brings out the potential of evolutionary strategies in optimizing model merging, paving the way for scalable and cost-effective model deployment in low-resource environments.