This paper focuses on enhancing the accuracy of corporate carbon emission prediction by leveraging multi-source data fusion of financial data and ESG reports. Existing methods predominantly rely on traditional machine learning approaches and struggle with issues of data heterogeneity and redundancy, limiting the effective utilization of complementary multi-source information. To address these challenges, we propose a deep learning framework, Carbon Aware Fusion-based Emission Network (CAFE-Net), which incorporates a bidirectional dynamic cross-attention module. Through custom modules, CAFE-Net adaptively extracts critical information from both financial data and text reports and integrates multi-source information to mitigate redundancy. Experimental results on real-world datasets validate that the proposed method significantly outperforms traditional approaches in predictive accuracy while demonstrating its practical applicability.