KDTL: knowledge-distilled transfer learning framework for diagnosing mental disorders using EEG spectrograms

Electroencephalography (EEG) is a well-known modality in neuroscience and is widely used in identifying and classifying neurological disorders. This paper investigates how EEG data can be used along with knowledge distillation-based deep learning models to detect mental disorders like epilepsy and sleep disorders. The EEG signals are converted into time–frequency plots using short-time Fourier transforms. Further, we propose a novel methodology for using knowledge distillation-based transfer learning (KDTL). Knowledge distillation is becoming quite prevalent in the machine learning field and is associated with various applications in our work; we propose its use in the detection of mental disorders from EEG spectrograms. We convert the EEGs using short-term Fourier transform to obtain time–frequency representation and apply teacher-student by first training a large teacher model and use knowledge distillation to train a student model. In our experiments, we found that ConvNext teacher and MobileNet student combination obtained better results. Our proposed KDTL approach is tested on two datasets with multiple cases, namely the Bonn and ISRUC datasets and obtain 98% and 95% accuracies, respectively. Further experimental results show that the overall KDTL methodology can obtain high classification accuracy across both datasets in binary and multiclass classifications and proves to be better than multiple prior works. Further, our KDTL approach provides a way to train lightweight models which have a smaller number of trainable parameters and thereby constitute lower training time overall. Our proposed KDTL-based approach obtained accurate results in diagnosing mental disorders from EEG spectrograms. Compared to other related methods, KDTL outperformed across tasks with obtained good results in both binary and multiclass classifications.