<p>Ocular diseases have emerged as the leading causes of blindness and low vision, necessitating timely detection and treatment. However, computer-aided approaches face significant challenges in accurately diagnosing these diseases. Specifically, ocular diseases often exhibit a long-tailed distribution, leading to a complex class-imbalanced scenario. Moreover, the coexistence of multiple diseases in a single patient gives rise to a problematic issue of label co-occurrence. In this study, we propose a novel alternate group training strategy as an effective approach to tackle the multi-label long-tailed data distribution problem. Firstly, we partition the long-tailed data into several groups based on semantic feature relations. This division helps reduce the challenges of class imbalance and label co-occurrence. With these groups established, we employ a gradient-based self-weighted loss to train a teacher network in an alternate way. Furthermore, a student model is trained on the original dataset under the guidance of the teacher network, utilizing a weighted class-balanced distillation loss. The class-balanced distillation loss also alleviates the class-wise imbalanced distribution and instance-wise label co-occurrence. Extensive experimental results have demonstrated the superiority of our proposed method which achieves promising performance on the publicly available dataset. In addition, our approach achieves promising performance when expanding the single-teacher model to multiple-teacher models.</p>