Chronic Kidney Disease (CKD) is a progressive condition that, without timely intervention, can lead to end-stage renal failure requiring dialysis or transplantation. Accurate multi-stage classification of longitudinal CKD data is critical for early referral and treatment. However, expert-labeled data is often scarce and costly. In practice, two types of datasets are common: a large, rule-based set labeled using eGFR, and a smaller, more reliable set annotated by GPs using broader clinical observations. In this study, we examine whether noisy but abundant eGFR labels and accurate GP labels provide complementary signals and whether combining them improves five-class CKD stage classification. We evaluate several strategies, including pre-training on eGFR data, fine-tuning on GP data, and hybrid approaches. We propose a fusion-based method that integrates latent representations from both datasets. Across various encoder architectures (LSTM, Bi-LSTM, Transformer, CNN+LSTM, CNN+Bi-LSTM, CNN+Transformer, TCN, TCN+LSTM, TCN+Bi-LSTM, TCN+Transformer), we test on the GP-labeled set. Our fusion method consistently outperforms baselines, supporting the hypothesis of complementary signals and demonstrating that fusion improves performance. This provides a practical solution for clinical settings with limited expert labels but abundant, noisy pseudo-labels.