This paper introduces an innovative approach to Medical Vision-Language Pre-training (Med-VLP) area in the specialized context of radiograph representation learning. While conventional methods frequently merge textual annotations into unified “reports”, we acknowledge the intrinsic hierarchical relationship between the “findings” and “impression” section in radiograph datasets. To establish a targeted correspondence between images and texts, we propose a novel HybridMED framework to align global-level visual representations with “impression” and token-level visual representations with “findings”. Moreover, our framework incorporates a generation decoder that employs two proxy tasks, responsible for generating the “impression” from (1) images, via a captioning branch, and (2) “findings”, through a summarization branch. Additionally, knowledge distillation is leveraged to facilitate the training process. Experiments on the MIMIC-CXR dataset reveal that our summarization branch effectively distills knowledge to the captioning branch, enhancing model performance without significantly increasing parameter requirements due to the shared self-attention and feed-forward architecture.