MMTSleepNet: a multi-modal fusion network via temporal sequence for sleep staging
摘要
Sleep staging is used to assist diagnosis of sleep disorders. Polysomnography (PSG) provides multiple channels of signal recording that reflects different working patterns of each sleep stage and can be utilized for automatic sleep staging. However, previous deep learning based sleep staging researches ignored the dependencies between local and global temporal sequences of multi-modal PSG signals. To solve this problem, this work proposed a multi-modal fusion network via temporal sequence for sleep staging (MMTSleepNet), consisting of local and global feature extraction (LGFE) block, adaptive modal feature recalibration (AMFR) block, multi-modal feature fusion (MFF) block and sleep staging block. LGFE block extracts both local and global contextual features to learn differentiated characteristics across sleep stages. AMFR block adaptively adjusts channel-wise weights on the concatenated feature maps to highlight more contributing PSG modalities, where each channel corresponds to a specific PSG signal or sub-signal. MFF block models feature dependencies to quantify the correlations between multi-modal temporal sequences. The proposed MMTSleepNet outperforms the state-of-the-art methods on Sleep-EDF-20, Sleep-EDF-78 and Sleep Heart Health Study (SHHS) datasets, with the accuracy rates of 87.5%, 82.9% and 87.8%, respectively.