In clinical diagnosis, medical corpora often comprise many numerical values, which poses a challenge for Large Language Models (LLMs) to make accurate decision. In order to evaluate the LLMs’ ability to reason with both non-numerical text and numerical sequences, we simulate real-world clinical scenarios and set up simplified survival prediction tasks, thereby developing Survival Prediction Dataset for COVID-19 (SPDC), which contains three datasets sampled under different conditions. Based on SPDC, we propose a highly adaptable framework using Concatenated Embedding of Non-Numerical Text and Numerical Sequences, which is denoted as CETS. Compared to conventional methods of processing plain text input, our framework embeds non-numerical text and numerical sequences separately, achieving a peak accuracy improvement of 4.32% and an average increase of 1.97% on SPDC. Through comparative experiments, we further clarify the impacts of standardization, patch length and stride, and the position embedding on the performance of CETS. As a reusable and easy-to-implement framework, CETS facilitates the performance of LLMs in processing clinical corpora and has extensive application potential in clinical medicine.