As the collection of medical data increases exponentially, there is a growing need for effective data processing and information extraction, particularly in high-risk areas like the ICU (Intensive Care Unit), where physicians need timely access of information for treatment decisions. This paper examines the effectiveness of large language models (LLMs) for automatically summarizing patients’ cases based on physician progress notes. We fine-tuned pre-trained language models on ICU progress notes to identify and summarize important medical issues. Our approach is innovative in its application of LLM fine-tuning specifically for ICU progress notes, incorporating knowledge augmentation and prompt engineering to enhance model performance. The experiments, conducted with real-world datasets, evaluated the models’ accuracy and efficiency using ROC AUC and Rouge-L F1 scores. The results demonstrated that LLMs could effectively extract critical medical issues, supporting clinical decision-making through automated summarization. This research highlights the potential of generative AI in improving the efficiency and accuracy of information extraction in life-critical environments.