Knowledge graph completion (KGC) seeks to infer absent facts within knowledge graphs (KGs). Graph Neural Networks (GNNs) have been demonstrated to be efficient for KGC. Several recent GNN-based methods utilize relational features to capture complex interactions between entities, achieving superior performance. However, previous approaches mostly rely on stacking numerous GNN layers to gather information of high-order neighborhood, which can lead to over-smoothing. In this paper, we propose a Contextualized Relational Graph Attention Network (CRGAT) that employs both local relation-aware and non-local graph context to update entities representations. Specifically, we develop a relation-aware aggregation component to capture information from one-hop neighbors by introducing parameters specific to each relation type. Moreover, CRGAT utilizes a biased random walk strategy to capture the non-local context and leverages Long Short-Term Memory (LSTM) networks to characterize relationships of entities and their non-local context. This method effectively captures multi-level information and complex structures by merging both local and non-local context with a gating mechanism. We compared CRGAT with several latest models across standard FB15k-237 and WN18RR data sets. Extensive experiments valid the performance of our approach.