Alzheimer’s disease (AD), a neurodegenerative disorder, significantly impairs the daily functioning of affected individuals and poses severe risks if not detected early, necessitating accurate and timely diagnosis. While previous research has predominantly relied on analyzing spontaneous speech patterns to extract linguistic, spectral, and prosodic features, either individually or in combination, there has been limited exploration of using pre-trained models to analyze all these speech features simultaneously. This gap underscores the necessity for innovative approaches that utilize the capabilities of pre-trained models to enhance the accuracy of AD diagnosis through comprehensive speech analysis. This paper presents a novel methodology that integrates pre-trained language transformers with a pre-trained vision transformer to enhance AD classification. The proposed approach facilitates a more comprehensive understanding of various speech aspects, aiding in the identification of patterns associated with dementia. Specifically, transformers such as BERT, ALBERT, Longformer, ELECTRA, ERNIE, RoBERTa, XLNet, and GPT-2 were explored, utilizing features including Mel-frequency cepstral coefficients (MFCC), Log-Mel spectrograms, and prosodic features. This integrated method offers a thorough analysis of speech characteristics pertinent to AD. The proposed methodology, rigorously validated using the ADReSS Challenge dataset, exhibits superior performance relative to current state-of-the-art models. Specifically, it achieves an accuracy of 90%, a recall of 91%, and a precision of 92% the highest by employing a combination of transformer models with Log-Mel (ViT), MFCC (ViT), and prosodic features. These results highlight the method’s significant potential to enhance AD detection through comprehensive multimodal analysis of speech and text.