The generation of radiology reports involves extracting features from medical images and converting them into textual descriptions. Transformer-based models have demonstrated outstanding performance. However, several challenges remain: the flattening operation on grid features may hinder precise lesion localization, and research on the limitations of these models in distinguishing medical terms from non-medical terms is still insufficient. To resolve these issues, this paper proposes a Spatial Information Enhancement Module (SIEM) that improves visual representations by integrating relative geometric features between grids. Additionally, we design an Adaptive Language-Vision Feature Fusion Module (ALVFF), which dynamically computes the importance of visual and language features for vocabulary prediction. We integrate these two modules into the memory-driven Transformer architecture, constructing the Adaptive Language and Vision feature fusion Network (ALV-Net) for radiology report generation. Experimental results on two radiology report datasets, IU-XRay and MIMIC-CXR, demonstrate that our method outperforms previous state-of-the-art models and achieves significant improvements in key performance metrics.