Sleep staging acts as a fundamental assessment to determine the sleep quality measurement and helps in diagnosing sleep disorders. To address the issues of data class imbalance and effective extraction in existing machine learning (ML) and deep learning (DL), a depth-sensitive attention based multi-model fusion encoder and decoder (DSAMED) system is proposed for effective sleep stage classification (SSC) which is inspired from the depth-sensitive attention and automatic multi-modal fusion (DSA2F) framework. In this work, the recordings of electro-oculogram (EOG) and electro-encephalogram (EEG) signals are considered whose multi-scale features are extracted using encoder. These multi-scale features are fused with multi-fusion model based attention mechanism (AM) that generates the multi-modal saliency features. The spatio-temporal (ST) features are captured using convolutional neural network (CNN) and restored with the low-layer details and mapped to achieve the prediction results. The proposed DSAMED’s efficiency is obtained with the classification accuracy of 98.44%, precision of 96.78%, and F1-score of 97.56% when compared to the conventional methods, spatial temporal sequential sleep scoring with bidirectional long short-term memory (STQS-Bi-LSTM) and time-related multi-modal sleep scoring (TRMSC).