Question Answering (QA) is a task that requires reasoning within natural language environments. Most existing works primarily utilize elaborately designed modules based on Graph Neural Networks (GNNs) and pre-trained Language Models (LMs) to perform joint reasoning over the Knowledge Graph (KG) and natural language text. However, these methods mainly focus on interactions between entity nodes and demonstrate a limited application of knowledge, failing to fully leverage the rich relational information within the KG. To alleviate these issues, we propose Relation-Aware Graph Reasoning Network (RAGN), aiming to leverage relational information to facilitate the reasoning process by jointly modeling knowledge triplets and knowledge entities. Specifically, to begin with, we propose Triplet-Token to capture diverse semantic relations between entities by encoding knowledge triplets in the KG. Furthermore, we propose a Relation-Aware Dynamic Aggregation (RADA) module designed to facilitate bidirectional interaction between knowledge entities and knowledge triplets by: (1) converting Triplet-Token into relation-specific attention biases and message features to update entity representations, while (2) refining triplet representations through their head/tail entities, thereby enabling a dynamic fusion of entity and relational information based on diverse semantic relationships. Finally, extensive experiments conducted on three knowledge-intensive QA benchmarks (CommonsenseQA, OpenBookQA, and MedQA-USMLE) demonstrate that RAGN outperforms traditional KG-augmented QA systems, validating the effectiveness of our proposed method.