In this paper, we present an enhanced approach for sexism identification that employs a Mixture of Transformers (MoT) framework, which leverages the language performance of individual models. Our method incorporates straightforward yet effective preprocessing modules, which are integrated with state-of-the-art Transformer architectures. We systematically compare the effectiveness of general-purpose, task-specific, and data source-specific models, evaluating both English and multilingual variants for the EXIST 2024 shared tasks. This refined approach addresses key limitations observed in earlier methods, leading to improved results across all evaluated tasks. Notably, our model demonstrates superior performance in soft-label evaluations compared to hard-label assessments. We introduce three distinct types of model mixtures, each of which achieved optimal results on training data and exhibited strong generalization to unseen data. Furthermore, the proposed architecture is designed for easy upgrades, maintains reasonable resource requirements, and delivers robust overall performance in competitive settings. Our findings underline the value of combining multiple Transformer models tailored to specific language and task requirements, paving the way for more adaptable and effective natural language processing solutions.