Sleep disorders are increasingly prevalent and substantially impact quality of life worldwide. Automatic sleep staging from electroencephalography (EEG) involves segmenting overnight recordings into physiologically defined stages and is central to the diagnosis and treatment of sleep-related disorders. We present a modular and computationally efficient framework that leverages EEG Pretrained Transformer (EEGPT) for EEG analysis, as a fixed feature extractor and pairs it with classical machine-learning classifiers. Our contribution lies in adapting and systematically evaluating EEGPT for the sleep-staging task, including its extension to multimodal inputs – electromyography (EMG), electrooculography (EOG), and electrocardiography (ECG). We show that coupling transformer-derived representations with lightweight tree-based models yields an efficient and accurate classifier. We systematically evaluate multiple transformer variants (pretrained and trained from scratch), classifier heads, and both EEG-only and multimodal settings on the modern Haaglanden Medisch Centrum (HMC) dataset. Our best configuration (EEGPT-little combined with a boosted Random Forest) achieves 74.91% accuracy under strict subject-level train-test separation. Overall, these results demonstrate that pretrained EEG transformers can serve as general-purpose feature extractors for downstream clinical tasks, offering a practical and data-efficient alternative to end-to-end neural pipelines.