In natural language processing, text summarization is an essential task, particularly for low-resource languages such as Vietnamese. Traditional techniques like LSTM (long short-term memory) networks demonstrated their potential efficiency in text summarizing; however, these techniques frequently have trouble collecting context and long-range dependencies. Thus, the authors propose T-LSTM, a hybrid model that benefits both LSTM networks and transformer structures, to address these issues. The LSTM component efficiently records sequential information, whereas the Transformer component demonstrates superior performance in managing long-range dependencies through self-attention processes. Our experiments show that T-LSTM shows ROUGE-1, ROUGE-2, and ROUGE-L have good performance, with scores of 63.34, 34.47, and 43.67 in configuration 1 and 62.88, 33.4, and 42.95 in configuration 2, respectively. There is also more cohesiveness in the generated summaries on the Vietnamese text summary task compared to both the standalone LSTM and Transformer models. The proposed model not only improves the quality of summarization but also integrates deep learning models with their complementary strengths. Experimental results indicate that T-LSTM presents a promising solution for enhancing text summarization in Vietnamese.