Cardiac surgery is associated with the risk of acute kidney injury (AKI), which can lead to prolonged hospital stays and increased mortality. Accurate prediction of AKI before its onset could significantly improve patient outcomes. However, existing AKI prediction models primarily focus on numerical features such as laboratory values and vital signs, while overlooking textual features, including preoperative diagnoses and surgical procedures. To address this limitation, we propose MedICL, which applies in-context learning (ICL) to the cardiac surgery domain. By leveraging the powerful comprehension and reasoning capabilities of large language models, MedICL enables the integration of textual and numerical features for AKI prediction. Nevertheless, the performance of ICL is highly sensitive to the quality of the provided examples, potentially limiting its effectiveness. To overcome this challenge, we introduce a Semantic Matching Unit (SMU), which selects semantically relevant examples for each sample, thereby significantly enhancing the model’s performance. Furthermore, we observed that ICL-based AKI predictions often suffer from instability and exhibit suboptimal performance on downstream tasks. To address these issues, we developed the Task Adaptability Enhancer (TAE), which calibrates the prediction probabilities generated by ICL on the validation set. This approach not only stabilizes the model’s outputs but also enhances its adaptability to specific task scenarios. A series of experiments on the datasets collected from West China Hospital (WCH) demonstrated that MedICL achieved state-of-the-art performance. These results highlight the indispensable role of medical text data in AKI prediction for cardiac surgery scenarios, showcasing its potential to improve clinical practice.