<p>Deep learning techniques have proven challenging to apply to medical imaging datasets due to their small size and domain-specific features. Significant advancements and increases in computing power have been made in medical image analysis. By tapping into deep features, we can enhance expert systems and intelligent tools, making diagnoses faster and less tedious. In this work, we present a novel technique for diagnosing soft tissue sarcomas using deep learning. To create our technique, we analyzed Magnetic Resonance (MR) images of patients with Leiomyosarcomas (LMS) and Liposarcomas (LPS). The objective is to determine the effects and improvements of deep learning on medical diagnostics. Transfer learning has emerged as a possible solution to this problem, especially when combined with transformer models that have already been trained. This study aims to evaluate the effectiveness of a novel hybrid model for medical image analysis, with a specific focus on image classification tasks, that combines newly developed optimization techniques with pre-trained transformers. As baseline methods, we evaluate the performance of the unique hybrid model against conventional convolutional neural network (CNN) models.</p>