Sleep is vital for physical recovery, brain function, and emotional health. Polysomnography (PSG) is the gold standard for assessing sleep quality, but it is intrusive and impractical for widespread application. Sound data is a non-intrusive alternative, but its complexity makes extracting meaningful information difficult. This research enhances sound-based sleep quality assessment using a multi-modal Knowledge Distillation (MKD) framework. The teacher model integrates PSG features, individual physical data, questionnaire responses, sleep stage sequences, and sound event features, using a Gated Variable Selection Neural Network (GVSN) based fusion network to identify key information from multi-modal inputs. The student model uses individual information and sound features extracted from one night’s sleep events and learns from the teacher model via a response-based MKD process. The results indicate that the student model’s accuracy improves significantly, highlighting the potential of MKD to enhance sound-based sleep quality assessment and analyze the impact of different modalities on the significance of the evaluation.