Multimodal Emotion Recognition in Conversation (MER) aims to identify emotions in conversation by integrating text, audio, and visual information, playing a crucial role in dialogue system, recommendation system and healthcare. Existing methods overlook the contribution discrepancy among modalities when integrating multimodal information, leading to the problem of imbalanced optimization, which in turn results in suboptimal performance. To address this problem, we propose a Debiased Hierarchical Knowledge Distillation (DHKD) framework that enhances weak modalities’ contribution and mitigates contribution discrepancy through innovative knowledge distillation, achieving a balanced optimization. As the major contribution in our proposed model, the debiased hierarchical knowledge distillation is designed to transfer the knowledge from the multimodality to the others at both the feature and logit levels. It boosts the contribution of single modalities, especially for these modalities with weak contribution. Extensive experiments on IEMOCAP and MELD datasets demonstrate the effectiveness of our approach.