Recent advancements in Large Vision Language Models (LVLMs) have achieved remarkable success in various vision-language downstream tasks. Following a uni-directional alignment framework, most LVLMs align vision encoders with the input space of Large Language Model (LLM) backbones. However, this paradigm overlooks alignment at the output end, leading to hallucinations. To address this issue, we propose a bi-directional semantic alignment framework for LVLMs training. In addition to alignment at the input level, the outputs of LVLMs are further aligned with visual signals through reinforcement learning. Specifically, we introduce Echo-Former to establish connections between input-level and output-level alignments, where it serves as a connection module as well as a reward model applicable to various LVLMs. Our method effectively reduces hallucinations and demonstrates efficacy in various benchmarks.