Text summarization has been an area of active research in the domain of natural language processing. Extractive text summarization focuses on highlighting the most essential parts of a document as its summary without any external additions to the original text. Conventional methods have focused on using sentence ranking algorithms like text rank, which uses position and similarity-based features to create graphs. However, the recent advent of deep learning, transfer learning, and context inclusion have been relatively undiscovered in this domain. This work proposes a graph-based ranking mechanism that uses relevant context to solve the task of extractive text summarization. Language modeling generates word representations that can provide a more comprehensive graph representation and achieve better results. The presented approach relies on graph-based ranking and the relevance and interpretation of each word in the document. The work is carried out on documents in the Marathi language with a standard-sized dataset. The proposed model has achieved a 75% F1 score and a Rouge L F1 score of 71%. The results obtained by the proposed system outperform those of state-of-the-art approaches. The system’s effectiveness is proved using the multiple performance metrics relevant to the text summarization task.